Sep 12 00:16:42.332000 ip-10-0-134-179 systemd[1]: Starting Kubernetes Kubelet...
Sep 12 00:16:42.705145 ip-10-0-134-179 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:42.705145 ip-10-0-134-179 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 12 00:16:42.705145 ip-10-0-134-179 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:42.705145 ip-10-0-134-179 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 12 00:16:42.705145 ip-10-0-134-179 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:42.708319 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.708231    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 12 00:16:42.713540 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713517    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.713540 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713535    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.713540 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713539    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.713540 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713543    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.713540 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713546    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713549    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713552    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713554    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713557    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713560    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713562    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713566    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713569    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713572    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713574    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713577    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713580    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713583    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713585    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713588    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713591    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713594    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713597    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713600    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.713750 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713602    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713605    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713608    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713610    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713613    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713617    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713621    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713631    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713635    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713639    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713642    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713644    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713647    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713650    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713654    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713657    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713659    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713662    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713665    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.714232 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713668    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713671    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713673    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713676    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713678    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713681    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713683    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713693    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713697    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713713    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713717    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713720    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713723    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713725    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713728    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713730    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713733    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713735    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713738    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713741    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.714697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713743    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713746    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713748    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713751    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713754    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713757    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713759    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713763    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713766    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713769    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713771    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713774    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713777    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713780    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713783    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713786    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713788    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713791    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713793    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.715220 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713796    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713806    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713809    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713811    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713814    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713816    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713819    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713822    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713824    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713826    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713829    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713832    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713834    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713837    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713839    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713842    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.713844    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715247    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715255    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715259    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.715682 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715262    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715265    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715268    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715271    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715276    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715279    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715282    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715284    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715287    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715289    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715292    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715295    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715297    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715299    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715309    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715312    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715314    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715317    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715319    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715322    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.716190 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715324    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715327    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715330    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715332    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715335    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715337    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715340    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715342    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715345    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715347    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715350    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715352    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715355    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715358    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715360    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715363    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715365    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715368    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715371    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.716774 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715374    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715376    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715379    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715381    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715384    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715388    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715391    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715395    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715405    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715408    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715410    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715413    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715415    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715418    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715420    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715423    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715425    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715428    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715430    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715433    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.717241 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715435    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715438    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715440    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715443    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715445    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715448    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715452    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715454    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715457    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715459    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715462    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715465    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715468    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715470    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715473    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715475    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715478    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715480    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715484    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715488    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.717746 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715491    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715499    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715502    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715505    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715508    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715510    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715513    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715516    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715518    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715521    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715523    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715526    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715528    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715531    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715533    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715536    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715539    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715971    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715977    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715980    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.718223 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715982    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715985    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715989    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715992    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.715996    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716000    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716003    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716006    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716009    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716012    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716014    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716018    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716022    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716031    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716034    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716037    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716040    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716043    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716045    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.718714 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716048    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716050    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716053    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716055    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716058    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716060    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716063    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716065    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716068    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716070    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716073    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716075    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716078    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716080    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716083    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716085    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716088    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716091    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716093    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716096    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.719194 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716098    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716101    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716103    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716106    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716109    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716112    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716122    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716130    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716132    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716135    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716137    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716140    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716142    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716145    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716147    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716150    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716152    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716155    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716157    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716160    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.719696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716162    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716166    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716169    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716171    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716174    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716176    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716179    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716185    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716188    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716190    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716193    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716196    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716199    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716201    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716204    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716207    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716210    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716212    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716215    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716217    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.720209 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716225    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716228    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716230    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716232    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716235    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716237    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716240    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716242    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716244    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716247    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716249    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716252    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716254    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716257    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716260    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716262    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.716265    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716336    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716369    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716375    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716380    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 12 00:16:42.720696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716387    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716390    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716394    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716398    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716401    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716404    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716408    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716411    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716414    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716417    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716420    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716423    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716427    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716430    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716434    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716437    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716440    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716443    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716446    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716450    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716453    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716456    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716459    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716462    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 12 00:16:42.721217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716465    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716468    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716471    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716474    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716478    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716481    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716484    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716487    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716491    2580 flags.go:64] FLAG: --enable-server="true"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716494    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716499    2580 flags.go:64] FLAG: --event-burst="100"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716502    2580 flags.go:64] FLAG: --event-qps="50"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716505    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716509    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716512    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716516    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716519    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716522    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716525    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716528    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716531    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716534    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716537    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716540    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716543    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 12 00:16:42.721867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716546    2580 flags.go:64] FLAG: --feature-gates=""
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716550    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716553    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716556    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716559    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716561    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716564    2580 flags.go:64] FLAG: --help="false"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716567    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716570    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716573    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716576    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716579    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716584    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716587    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716590    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716594    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716597    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716600    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716602    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716605    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716608    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716611    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716614    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716617    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 12 00:16:42.722502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716620    2580 flags.go:64] FLAG: --lock-file=""
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716622    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716625    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716628    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716633    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716637    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716640    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716642    2580 flags.go:64] FLAG: --logging-format="text"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716645    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716648    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716651    2580 flags.go:64] FLAG: --manifest-url=""
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716654    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716658    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716661    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716665    2580 flags.go:64] FLAG: --max-pods="110"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716667    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716670    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716673    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716676    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716679    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716683    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716686    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716693    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716697    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716715    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 12 00:16:42.723100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716720    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716725    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716730    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716734    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716737    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716740    2580 flags.go:64] FLAG: --port="10250"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716743    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716746    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b7fd181cb945f9c6"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716749    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716752    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716755    2580 flags.go:64] FLAG: --register-node="true"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716758    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716762    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716767    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716769    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716772    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716776    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716779    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716782    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716785    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716788    2580 flags.go:64] FLAG: --runonce="false"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716791    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716794    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716797    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716800    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 12 00:16:42.723771 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716803    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716806    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716809    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716814    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716817    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716821    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716824    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716827    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716834    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716837    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716840    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716845    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716848    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716851    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716855    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716858    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716860    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716863    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716866    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716869    2580 flags.go:64] FLAG: --v="2"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716873    2580 flags.go:64] FLAG: --version="false"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716878    2580 flags.go:64] FLAG: --vmodule=""
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716882    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.716885    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 12 00:16:42.724384 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717021    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717025    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717029    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717031    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717035    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717038    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717041    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717044    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717046    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717049    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717051    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717055    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717058    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717062    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717065    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717067    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717070    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717072    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717075    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.725023 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717078    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717081    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717083    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717086    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717089    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717091    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717094    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717097    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717099    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717102    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717104    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717108    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717110    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717113    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717116    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717118    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717121    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717124    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717126    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717129    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.725560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717131    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717134    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717136    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717139    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717143    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717146    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717151    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717155    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717158    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717161    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717164    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717168    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717172    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717175    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717178    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717180    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717183    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717186    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717188    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.726062 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717191    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717193    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717196    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717199    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717201    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717204    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717207    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717209    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717212    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717214    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717216    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717219    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717222    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717224    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717227    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717229    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717232    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717235    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717238    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717242    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.726523 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717245    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717248    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717251    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717253    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717256    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717259    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717261    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717264    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717266    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717269    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717272    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717274    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717277    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717281    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717284    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717287    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717290    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717293    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717296    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717298    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.727024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.717301    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.717901    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725070    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725086    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725105    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725110    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725174    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725179    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725183    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725186    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725189    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725192    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725195    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725198    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.727520 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725201    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725203    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725206    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725210    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725215    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725218    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725221    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725224    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725227    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725230    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725232    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725235    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725238    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725241    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725243    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725246    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725249    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725253    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725255    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.727955 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725258    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725261    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725264    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725267    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725270    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725273    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725276    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725279    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725282    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725285    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725287    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725290    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725292    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725295    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725298    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725300    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725304    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725306    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725309    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725311    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.728413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725314    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725317    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725319    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725322    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725324    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725328    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725330    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725333    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725336    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725338    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725342    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725345    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725348    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725350    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725354    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725358    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725360    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725363    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725366    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.728961 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725369    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725371    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725374    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725377    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725379    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725382    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725384    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725387    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725389    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725392    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725395    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725398    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725400    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725403    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725406    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725408    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725411    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725414    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725417    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.729424 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725421    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725424    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725427    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725431    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725433    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725441    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725445    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725447    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725450    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725453    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725456    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725458    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725461    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725464    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725469    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:42.729897 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725598    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725602    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725605    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725609    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725611    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725614    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725617    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725620    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725624    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725628    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725631    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725634    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725637    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725640    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725643    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725646    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725649    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725652    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725655    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:42.730264 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725657    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725660    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725664    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725667    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725670    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725673    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725675    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725678    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725680    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725683    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725685    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725688    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725690    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725693    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725695    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725699    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725714    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725717    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725720    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725723    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:42.730728 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725726    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725728    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725731    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725734    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725737    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725739    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725742    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725745    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725747    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725750    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725753    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725756    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725758    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725761    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725764    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725766    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725769    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725772    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725775    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725777    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:42.731210 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725780    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725782    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725785    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725788    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725790    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725793    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725795    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725798    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725800    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725803    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725806    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725808    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725811    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725813    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725816    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725819    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725821    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725824    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725827    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725829    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:42.731681 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725832    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725835    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725838    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725840    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725843    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725845    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725848    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725851    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725853    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725856    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725859    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725861    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725865    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725869    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725872    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725875    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725877    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725881    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725884    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:42.732182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:42.725886    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725891    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.725982    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.728296    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.729674    2580 server.go:1019] "Starting client certificate rotation"
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.729791    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.729824    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:16:42.732640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.732568    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:16:42.734332 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.734313    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:16:42.748041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.748019    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 12 00:16:42.751573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.751555    2580 log.go:25] "Validated CRI v1 image API"
Sep 12 00:16:42.752447 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.752422    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 12 00:16:42.756006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.755984    2580 fs.go:135] Filesystem UUIDs: map[57622068-6e90-4e7a-be9f-253de3ae52cd:/dev/nvme0n1p3 6099e6bb-6572-4ae4-b6c6-0932f75e2dab:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 12 00:16:42.756083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.756004    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 12 00:16:42.761722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.761692    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:16:42.761989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.761883    2580 manager.go:216] Machine: {Timestamp:2026-09-12 00:16:42.760398904 +0000 UTC m=+0.320450455 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3165662 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0307fd459a7d023d4a7e1a1eabbb SystemUUID:ec2d0307-fd45-9a7d-023d-4a7e1a1eabbb BootID:54ec8393-d7ac-4658-be39-e510321db037 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:02:1f:88:5a:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:02:1f:88:5a:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:6d:8c:6a:72:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 12 00:16:42.761989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.761984    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 12 00:16:42.762088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.762064    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 12 00:16:42.763190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.763169    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 12 00:16:42.763324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.763193    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-179.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 12 00:16:42.763964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.763954    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 12 00:16:42.763996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.763967    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 12 00:16:42.763996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.763975    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:16:42.764757 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.764747    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:16:42.765784 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.765774    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 12 00:16:42.769120 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769108    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:16:42.769241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769232    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 12 00:16:42.769279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769262    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 12 00:16:42.769279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769278    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 12 00:16:42.769331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769300    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 12 00:16:42.769331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769312    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 12 00:16:42.769331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769327    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 12 00:16:42.769773 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769698    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:16:42.769832 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.769788    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:16:42.771662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.771637    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 12 00:16:42.774365 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.774350    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 12 00:16:42.774420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.774393    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 12 00:16:42.775902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775885    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 12 00:16:42.775902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775902    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 12 00:16:42.775902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775908    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775914    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775923    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775928    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775941    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775946    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775953    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775959    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775973    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 12 00:16:42.776044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.775978    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 12 00:16:42.776647 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.776636    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 12 00:16:42.776679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.776647    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 12 00:16:42.778849 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.778816    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-179.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 12 00:16:42.778849 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.778837    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-179.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 12 00:16:42.778941 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.778845    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 12 00:16:42.780546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.780533    2580 server.go:1268] "Started kubelet"
Sep 12 00:16:42.780683 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.780660    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 12 00:16:42.780746 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.780681    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 12 00:16:42.780784 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.780761    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 12 00:16:42.784040 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.784015    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 12 00:16:42.784361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.784340    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vq89p"
Sep 12 00:16:42.784670 ip-10-0-134-179 systemd[1]: Started Kubernetes Kubelet.
Sep 12 00:16:42.785085 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.785066    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 12 00:16:42.786199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.786178    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 12 00:16:42.789920 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.789902    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vq89p"
Sep 12 00:16:42.792506 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.792485    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 12 00:16:42.793002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.792989    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 12 00:16:42.793079 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.793057    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:42.793657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793526    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 12 00:16:42.793657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793530    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 12 00:16:42.793840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793663    2580 factory.go:55] Registering systemd factory
Sep 12 00:16:42.793840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793679    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 12 00:16:42.793840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793665    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 12 00:16:42.793840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793773    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 12 00:16:42.793840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793785    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 12 00:16:42.794081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793884    2580 factory.go:153] Registering CRI-O factory
Sep 12 00:16:42.794081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793901    2580 factory.go:223] Registration of the crio container factory successfully
Sep 12 00:16:42.794081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793953    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 12 00:16:42.794081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.793972    2580 factory.go:103] Registering Raw factory
Sep 12 00:16:42.794081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.794003    2580 manager.go:1196] Started watching for new ooms in manager
Sep 12 00:16:42.794337 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.794324    2580 manager.go:349] Starting recovery of all containers
Sep 12 00:16:42.795193 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.794080    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-179.ec2.internal.18d46aa394bc2ece  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-179.ec2.internal,UID:ip-10-0-134-179.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-179.ec2.internal,},FirstTimestamp:2026-09-12 00:16:42.780503758 +0000 UTC m=+0.340555309,LastTimestamp:2026-09-12 00:16:42.780503758 +0000 UTC m=+0.340555309,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-179.ec2.internal,}"
Sep 12 00:16:42.802295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.802273    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:42.804769 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.804745    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-179.ec2.internal\" not found" node="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:42.804840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.804821    2580 manager.go:354] Recovery completed
Sep 12 00:16:42.808869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.808832    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:42.810380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.810365    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:42.810441 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.810397    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:42.810441 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.810411    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:42.811435 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.811420    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 12 00:16:42.811435 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.811433    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 12 00:16:42.811563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.811452    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:16:42.811563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.811464    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:42.812161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812142    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:42.812230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812171    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:42.812230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812181    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:42.812693 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812680    2580 policy_none.go:49] "None policy: Start"
Sep 12 00:16:42.812801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812698    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 12 00:16:42.812801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.812745    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 12 00:16:42.813677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.813665    2580 policy_none.go:47] "Start"
Sep 12 00:16:42.853291 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853276    2580 manager.go:337] "Starting Device Plugin manager"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.853310    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853320    2580 server.go:87] "Starting device plugin registration server"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853624    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853648    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853819    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853901    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.853910    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.854235    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 12 00:16:42.858310 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.854285    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:42.940138 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.940106    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 12 00:16:42.941302 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.941282    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 12 00:16:42.941409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.941339    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 12 00:16:42.941409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.941371    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 12 00:16:42.941488 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.941417    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 12 00:16:42.943720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.943683    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:42.954522 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.954506    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:42.956336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.956259    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:42.956336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.956295    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:42.956336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.956310    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:42.956336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.956338    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:42.962870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:42.962853    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:42.962948 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.962875    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-179.ec2.internal\": node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:42.979260 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:42.979238    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:43.041907 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.041856    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"]
Sep 12 00:16:43.042036 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.041983    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:43.043618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.043600    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:43.043734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.043631    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:43.043734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.043641    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:43.044781 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.044769    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:43.044931 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.044915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.044966 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.044948    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:43.045673 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045660    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:43.045762 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045685    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:43.045762 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045694    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:43.045762 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045700    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:43.045762 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045749    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:43.045895 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.045766    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:43.046966 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.046951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.047014 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.046985    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:43.047725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.047684    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:43.047786 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.047729    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:43.047786 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.047744    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:43.072290 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.072272    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-179.ec2.internal\" not found" node="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.076542 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.076526    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-179.ec2.internal\" not found" node="ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.079568 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.079555    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:43.096258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.096243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.096324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.096267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.096324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.096284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f84cc7b6dd936cd66de2a1a58b0376d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-179.ec2.internal\" (UID: \"9f84cc7b6dd936cd66de2a1a58b0376d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.179952 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.179928    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:43.196612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.196699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.196699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f84cc7b6dd936cd66de2a1a58b0376d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-179.ec2.internal\" (UID: \"9f84cc7b6dd936cd66de2a1a58b0376d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.196812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f84cc7b6dd936cd66de2a1a58b0376d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-179.ec2.internal\" (UID: \"9f84cc7b6dd936cd66de2a1a58b0376d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.196812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.196812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.196749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/554e5250205817bcdbdb60ad48e7d590-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal\" (UID: \"554e5250205817bcdbdb60ad48e7d590\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.280081 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.280049    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:43.375587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.375531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:43.380600 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:43.380513    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:43.380600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:43.380540    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:44.906964 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:44.906922    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.007489 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.007404    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.108118 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.108078    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.155552 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.155521    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 12 00:16:45.155996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.155652    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 00:16:45.155996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.155686    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 00:16:45.208186 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.208158    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.219078 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.219044    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-11 00:11:44 +0000 UTC" deadline="2028-06-17 07:05:45.540999326 +0000 UTC"
Sep 12 00:16:45.219078 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.219076    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15462h49m0.321926193s"
Sep 12 00:16:45.308385 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.308352    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.308710 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:45.308682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f84cc7b6dd936cd66de2a1a58b0376d.slice/crio-08b701716c077034f92a488b1d85ba95638a0f6a23fea46f602861cf1b0eb754 WatchSource:0}: Error finding container 08b701716c077034f92a488b1d85ba95638a0f6a23fea46f602861cf1b0eb754: Status 404 returned error can't find the container with id 08b701716c077034f92a488b1d85ba95638a0f6a23fea46f602861cf1b0eb754
Sep 12 00:16:45.309092 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:45.309067    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554e5250205817bcdbdb60ad48e7d590.slice/crio-f0ad2235ca9aef15e11072e57af39f99b30b954c7a8b113a2f7b0efbbd07b358 WatchSource:0}: Error finding container f0ad2235ca9aef15e11072e57af39f99b30b954c7a8b113a2f7b0efbbd07b358: Status 404 returned error can't find the container with id f0ad2235ca9aef15e11072e57af39f99b30b954c7a8b113a2f7b0efbbd07b358
Sep 12 00:16:45.313680 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.313667    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:16:45.370777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.370726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal" event={"ID":"554e5250205817bcdbdb60ad48e7d590","Type":"ContainerStarted","Data":"f0ad2235ca9aef15e11072e57af39f99b30b954c7a8b113a2f7b0efbbd07b358"}
Sep 12 00:16:45.371820 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.371797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal" event={"ID":"9f84cc7b6dd936cd66de2a1a58b0376d","Type":"ContainerStarted","Data":"08b701716c077034f92a488b1d85ba95638a0f6a23fea46f602861cf1b0eb754"}
Sep 12 00:16:45.408990 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.408961    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.509468 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.509423    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.610163 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:45.610093    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-179.ec2.internal\" not found"
Sep 12 00:16:45.610986 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.610973    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:45.619891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.619867    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:45.634273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.634254    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 00:16:45.635859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.635847    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal"
Sep 12 00:16:45.644398 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.644378    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 00:16:45.809255 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:45.809215    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:46.105058 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.105026    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:46.196388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.196360    2580 apiserver.go:52] "Watching apiserver"
Sep 12 00:16:46.205406 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.205381    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 12 00:16:46.205954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.205918    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vmbg5","openshift-multus/multus-dlmjc","openshift-network-diagnostics/network-check-target-6ztlj","openshift-network-operator/iptables-alerter-f46p2","kube-system/global-pull-secret-syncer-72whv","kube-system/konnectivity-agent-lxhxh","openshift-dns/node-resolver-5vv4d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal","openshift-multus/network-metrics-daemon-jhbcl","openshift-ovn-kubernetes/ovnkube-node-42sn5","kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59","openshift-cluster-node-tuning-operator/tuned-t7lz5","openshift-image-registry/node-ca-jnf7d"]
Sep 12 00:16:46.208267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.208248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.210140 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.210120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.210411 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.210389    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.210502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.210392    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.210502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.210395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tp4kk\""
Sep 12 00:16:46.212099 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.212079    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jg8qq\""
Sep 12 00:16:46.212188 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.212134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:46.212248 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.212210    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:46.212336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.212317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.212550 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.212536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.214218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.214197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.216314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.216295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.216399 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.216367    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:46.217579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.217287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.217579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.217397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ltmqg\""
Sep 12 00:16:46.217579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.217469    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 12 00:16:46.217579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.217483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.218625 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.218577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.219099 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.219082    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 12 00:16:46.220899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.220558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.220899 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.220627    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:46.220899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.220737    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-djnt9\""
Sep 12 00:16:46.220899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.220737    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 12 00:16:46.221140 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.220985    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 12 00:16:46.223157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.223137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.225037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.225376 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 12 00:16:46.225376 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.225519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 12 00:16:46.225754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-th2hh\""
Sep 12 00:16:46.225754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.225627    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 12 00:16:46.226040 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.226020    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.227103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.227037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 12 00:16:46.227334 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.227315    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dmmn8\""
Sep 12 00:16:46.227751 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.227732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.230031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.229862    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 12 00:16:46.230031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.229892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q69j2\""
Sep 12 00:16:46.230177 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.230037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 12 00:16:46.233916 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.233891    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 12 00:16:46.234161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.234146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.234385 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.234365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.235510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.235488    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.235836    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.236187    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.236530    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.236555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8zxtt\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.236833    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.236839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwsxc\" (UniqueName: \"kubernetes.io/projected/589333b5-36cc-4fc7-99a3-7ab405c26767-kube-api-access-jwsxc\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d75218-5ef8-477b-930e-0f3cac2806af-tmp-dir\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn8kw\" (UniqueName: \"kubernetes.io/projected/7a7bca5f-401d-4c76-ac5f-305a5d176b80-kube-api-access-fn8kw\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-lib-modules\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-modprobe-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-host\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-tuned\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5p4r\" (UniqueName: \"kubernetes.io/projected/21d75218-5ef8-477b-930e-0f3cac2806af-kube-api-access-g5p4r\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-kubelet-config\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-run\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-sys\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-var-lib-kubelet\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52f1aae7-285c-4898-be26-2e26dc23bcc2-konnectivity-ca\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.238941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysconfig\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.237985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-conf\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-systemd\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a7bca5f-401d-4c76-ac5f-305a5d176b80-iptables-alerter-script\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a7bca5f-401d-4c76-ac5f-305a5d176b80-host-slash\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52f1aae7-285c-4898-be26-2e26dc23bcc2-agent-certs\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.238525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21d75218-5ef8-477b-930e-0f3cac2806af-hosts-file\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-kubernetes\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-tmp\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmg8x\" (UniqueName: \"kubernetes.io/projected/d4e849ab-ad3f-4296-b327-42951f5b82ab-kube-api-access-bmg8x\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-dbus\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.239812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.239632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.240605 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.240586    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b7wkj\""
Sep 12 00:16:46.240875 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.240856    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 12 00:16:46.240994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.240977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 12 00:16:46.241065 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.240849    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 12 00:16:46.255535 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.255515    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5krgz"
Sep 12 00:16:46.265087 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.265069    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5krgz"
Sep 12 00:16:46.320901 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.320878    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 12 00:16:46.340718 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e50127b-bff1-431e-aa20-3240d34fbc0b-serviceca\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-socket-dir-parent\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-kubelet\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-etc-kubernetes\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5fhf\" (UniqueName: \"kubernetes.io/projected/1ecadca1-f699-49ea-9329-0c921a042679-kube-api-access-r5fhf\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d75218-5ef8-477b-930e-0f3cac2806af-tmp-dir\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.340885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn8kw\" (UniqueName: \"kubernetes.io/projected/7a7bca5f-401d-4c76-ac5f-305a5d176b80-kube-api-access-fn8kw\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.341180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.341180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e50127b-bff1-431e-aa20-3240d34fbc0b-host\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.341180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.340954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-conf-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.341180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.341180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-device-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d75218-5ef8-477b-930e-0f3cac2806af-tmp-dir\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-log-socket\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-netd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7385b79c-9e06-4e15-a915-d043b242415c-ovn-node-metrics-cert\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-tuned\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.341445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5p4r\" (UniqueName: \"kubernetes.io/projected/21d75218-5ef8-477b-930e-0f3cac2806af-kube-api-access-g5p4r\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-kubelet-config\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-run\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-systemd\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-systemd-units\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-var-lib-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52f1aae7-285c-4898-be26-2e26dc23bcc2-konnectivity-ca\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-multus-certs\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a7bca5f-401d-4c76-ac5f-305a5d176b80-iptables-alerter-script\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52f1aae7-285c-4898-be26-2e26dc23bcc2-agent-certs\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-kubelet\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.341831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-etc-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-env-overrides\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-multus-daemon-config\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.341972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21d75218-5ef8-477b-930e-0f3cac2806af-hosts-file\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-tmp\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmg8x\" (UniqueName: \"kubernetes.io/projected/d4e849ab-ad3f-4296-b327-42951f5b82ab-kube-api-access-bmg8x\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342095    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-dbus\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-kubelet-config\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-bin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-run\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.342593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmdhg\" (UniqueName: \"kubernetes.io/projected/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kube-api-access-nmdhg\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-systemd\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwsxc\" (UniqueName: \"kubernetes.io/projected/589333b5-36cc-4fc7-99a3-7ab405c26767-kube-api-access-jwsxc\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-lib-modules\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-config\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-cni-binary-copy\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-modprobe-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-slash\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-script-lib\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svz9\" (UniqueName: \"kubernetes.io/projected/7385b79c-9e06-4e15-a915-d043b242415c-kube-api-access-7svz9\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws4qj\" (UniqueName: \"kubernetes.io/projected/fe313196-f058-4a7a-9de7-6307e39a7d4a-kube-api-access-ws4qj\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-host\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-sys\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f79f9dca-61f6-45f2-b1e5-248e521d599f-dbus\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-var-lib-kubelet\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.343383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysconfig\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-conf\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-ovn\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-bin\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-lib-modules\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.342975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsrk2\" (UniqueName: \"kubernetes.io/projected/6e50127b-bff1-431e-aa20-3240d34fbc0b-kube-api-access-gsrk2\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a7bca5f-401d-4c76-ac5f-305a5d176b80-host-slash\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-systemd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-node-log\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-system-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-kubernetes\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-modprobe-d\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52f1aae7-285c-4898-be26-2e26dc23bcc2-konnectivity-ca\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a7bca5f-401d-4c76-ac5f-305a5d176b80-iptables-alerter-script\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-netns\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.344123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-cnibin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21d75218-5ef8-477b-930e-0f3cac2806af-hosts-file\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.343411    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-multus\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.343444    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-system-cni-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-var-lib-kubelet\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.343595    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:16:46.843562264 +0000 UTC m=+2.977552472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-host\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a7bca5f-401d-4c76-ac5f-305a5d176b80-host-slash\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysctl-conf\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-kubernetes\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-sys\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-sysconfig\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-cnibin\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-k8s-cni-cncf-io\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-netns\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.344978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.343982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-hostroot\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.345789 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.344003    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:46.843985963 +0000 UTC m=+2.977976179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:46.346619 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.346479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-etc-tuned\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.346706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.346510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e849ab-ad3f-4296-b327-42951f5b82ab-tmp\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.346746 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.346722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52f1aae7-285c-4898-be26-2e26dc23bcc2-agent-certs\") pod \"konnectivity-agent-lxhxh\" (UID: \"52f1aae7-285c-4898-be26-2e26dc23bcc2\") " pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.348951 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.348930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn8kw\" (UniqueName: \"kubernetes.io/projected/7a7bca5f-401d-4c76-ac5f-305a5d176b80-kube-api-access-fn8kw\") pod \"iptables-alerter-f46p2\" (UID: \"7a7bca5f-401d-4c76-ac5f-305a5d176b80\") " pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.349759 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.349742    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:46.349841 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.349778    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:46.349841 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.349793    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:46.349968 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.349908    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:46.849891247 +0000 UTC m=+2.983881469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:46.350355 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.350335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5p4r\" (UniqueName: \"kubernetes.io/projected/21d75218-5ef8-477b-930e-0f3cac2806af-kube-api-access-g5p4r\") pod \"node-resolver-5vv4d\" (UID: \"21d75218-5ef8-477b-930e-0f3cac2806af\") " pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.350618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.350592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwsxc\" (UniqueName: \"kubernetes.io/projected/589333b5-36cc-4fc7-99a3-7ab405c26767-kube-api-access-jwsxc\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.352056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.352034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmg8x\" (UniqueName: \"kubernetes.io/projected/d4e849ab-ad3f-4296-b327-42951f5b82ab-kube-api-access-bmg8x\") pod \"tuned-t7lz5\" (UID: \"d4e849ab-ad3f-4296-b327-42951f5b82ab\") " pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.445206 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-netns\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.445206 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-cnibin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-multus\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-system-cni-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-cnibin\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-netns\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-k8s-cni-cncf-io\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-k8s-cni-cncf-io\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-netns\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-cnibin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-hostroot\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-multus\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-hostroot\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e50127b-bff1-431e-aa20-3240d34fbc0b-serviceca\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.445461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-netns\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-socket-dir-parent\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-kubelet\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-etc-kubernetes\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5fhf\" (UniqueName: \"kubernetes.io/projected/1ecadca1-f699-49ea-9329-0c921a042679-kube-api-access-r5fhf\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e50127b-bff1-431e-aa20-3240d34fbc0b-host\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-conf-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-device-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-log-socket\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-netd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e50127b-bff1-431e-aa20-3240d34fbc0b-serviceca\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7385b79c-9e06-4e15-a915-d043b242415c-ovn-node-metrics-cert\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-system-cni-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.446096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-cnibin\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-systemd-units\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-systemd-units\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-var-lib-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-var-lib-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-multus-certs\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.445996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-run-multus-certs\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-kubelet\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-kubelet\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-socket-dir-parent\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-etc-kubernetes\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-kubelet\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-device-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.446859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-log-socket\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-run-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-netd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e50127b-bff1-431e-aa20-3240d34fbc0b-host\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-conf-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-etc-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-env-overrides\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-multus-daemon-config\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.446986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe313196-f058-4a7a-9de7-6307e39a7d4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-bin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.447631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmdhg\" (UniqueName: \"kubernetes.io/projected/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kube-api-access-nmdhg\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-config\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-cni-binary-copy\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-slash\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-script-lib\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svz9\" (UniqueName: \"kubernetes.io/projected/7385b79c-9e06-4e15-a915-d043b242415c-kube-api-access-7svz9\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-etc-openvswitch\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws4qj\" (UniqueName: \"kubernetes.io/projected/fe313196-f058-4a7a-9de7-6307e39a7d4a-kube-api-access-ws4qj\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-multus-daemon-config\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-host-var-lib-cni-bin\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-env-overrides\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-ovn\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-bin\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.448414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsrk2\" (UniqueName: \"kubernetes.io/projected/6e50127b-bff1-431e-aa20-3240d34fbc0b-kube-api-access-gsrk2\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-systemd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.447980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-node-log\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-system-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-system-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ecadca1-f699-49ea-9329-0c921a042679-multus-cni-dir\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fe313196-f058-4a7a-9de7-6307e39a7d4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-ovn\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-cni-bin\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448265    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-host-slash\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-run-systemd\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7385b79c-9e06-4e15-a915-d043b242415c-node-log\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ecadca1-f699-49ea-9329-0c921a042679-cni-binary-copy\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.448998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-config\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.449043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7385b79c-9e06-4e15-a915-d043b242415c-ovnkube-script-lib\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.449191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.449136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7385b79c-9e06-4e15-a915-d043b242415c-ovn-node-metrics-cert\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.454692 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.454663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5fhf\" (UniqueName: \"kubernetes.io/projected/1ecadca1-f699-49ea-9329-0c921a042679-kube-api-access-r5fhf\") pod \"multus-dlmjc\" (UID: \"1ecadca1-f699-49ea-9329-0c921a042679\") " pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.456372 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.456350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmdhg\" (UniqueName: \"kubernetes.io/projected/1c33dd74-8ecb-4929-b91c-4d95bda4aa8c-kube-api-access-nmdhg\") pod \"aws-ebs-csi-driver-node-fdq59\" (UID: \"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.456970 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.456926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws4qj\" (UniqueName: \"kubernetes.io/projected/fe313196-f058-4a7a-9de7-6307e39a7d4a-kube-api-access-ws4qj\") pod \"multus-additional-cni-plugins-vmbg5\" (UID: \"fe313196-f058-4a7a-9de7-6307e39a7d4a\") " pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.457361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.457340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svz9\" (UniqueName: \"kubernetes.io/projected/7385b79c-9e06-4e15-a915-d043b242415c-kube-api-access-7svz9\") pod \"ovnkube-node-42sn5\" (UID: \"7385b79c-9e06-4e15-a915-d043b242415c\") " pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.457550 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.457529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsrk2\" (UniqueName: \"kubernetes.io/projected/6e50127b-bff1-431e-aa20-3240d34fbc0b-kube-api-access-gsrk2\") pod \"node-ca-jnf7d\" (UID: \"6e50127b-bff1-431e-aa20-3240d34fbc0b\") " pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.519410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.519376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5vv4d"
Sep 12 00:16:46.527683 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.527661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7lz5"
Sep 12 00:16:46.542234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.542211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f46p2"
Sep 12 00:16:46.548777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.548737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:16:46.555698 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.555678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vmbg5"
Sep 12 00:16:46.562374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.562353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dlmjc"
Sep 12 00:16:46.570911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.570895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:16:46.576513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.576496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jnf7d"
Sep 12 00:16:46.581314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.581297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59"
Sep 12 00:16:46.624594 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.624564    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:46.850706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.850640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:46.850706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.850672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:46.850706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:46.850699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:46.850908 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850808    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:46.850908 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850869    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:16:47.850855246 +0000 UTC m=+3.984845452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:46.850908 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850809    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:46.851006 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850933    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:47.850920732 +0000 UTC m=+3.984910934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:46.851006 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850816    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:46.851006 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850956    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:46.851006 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850966    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:46.851006 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:46.850996    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:47.850987191 +0000 UTC m=+3.984977394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:47.238926 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.238899    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e849ab_ad3f_4296_b327_42951f5b82ab.slice/crio-118bf7ca8dacc8ed2e689d672e3fea51ec1bc76f118c6dc6b0e583ccc4cad903 WatchSource:0}: Error finding container 118bf7ca8dacc8ed2e689d672e3fea51ec1bc76f118c6dc6b0e583ccc4cad903: Status 404 returned error can't find the container with id 118bf7ca8dacc8ed2e689d672e3fea51ec1bc76f118c6dc6b0e583ccc4cad903
Sep 12 00:16:47.241553 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.241522    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d75218_5ef8_477b_930e_0f3cac2806af.slice/crio-7036e40250e5afd4971a99361ad893350f081ddf9ce74dddc851cce2e1e7884f WatchSource:0}: Error finding container 7036e40250e5afd4971a99361ad893350f081ddf9ce74dddc851cce2e1e7884f: Status 404 returned error can't find the container with id 7036e40250e5afd4971a99361ad893350f081ddf9ce74dddc851cce2e1e7884f
Sep 12 00:16:47.243840 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.243809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e50127b_bff1_431e_aa20_3240d34fbc0b.slice/crio-fd5b4c6c1b364ec1bdbae07d720d0742e8c9f5a98209777e71e930fd1810a4dc WatchSource:0}: Error finding container fd5b4c6c1b364ec1bdbae07d720d0742e8c9f5a98209777e71e930fd1810a4dc: Status 404 returned error can't find the container with id fd5b4c6c1b364ec1bdbae07d720d0742e8c9f5a98209777e71e930fd1810a4dc
Sep 12 00:16:47.244571 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.244549    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7385b79c_9e06_4e15_a915_d043b242415c.slice/crio-ede0ef9ed67d3ff9b43d28b6297be88e3a641ae5d8e53fe96a0d75a2834ef3fc WatchSource:0}: Error finding container ede0ef9ed67d3ff9b43d28b6297be88e3a641ae5d8e53fe96a0d75a2834ef3fc: Status 404 returned error can't find the container with id ede0ef9ed67d3ff9b43d28b6297be88e3a641ae5d8e53fe96a0d75a2834ef3fc
Sep 12 00:16:47.245466 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.245351    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe313196_f058_4a7a_9de7_6307e39a7d4a.slice/crio-446e09c732c0c153329f748742993b8e58e87ab93d31b86b29d99d60ebaaf542 WatchSource:0}: Error finding container 446e09c732c0c153329f748742993b8e58e87ab93d31b86b29d99d60ebaaf542: Status 404 returned error can't find the container with id 446e09c732c0c153329f748742993b8e58e87ab93d31b86b29d99d60ebaaf542
Sep 12 00:16:47.246237 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.246150    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c33dd74_8ecb_4929_b91c_4d95bda4aa8c.slice/crio-f8e00c41aa5fe2e5f5fd0d13c3e06bfe9d5a422f40b5613c6a0f28f0a70e749b WatchSource:0}: Error finding container f8e00c41aa5fe2e5f5fd0d13c3e06bfe9d5a422f40b5613c6a0f28f0a70e749b: Status 404 returned error can't find the container with id f8e00c41aa5fe2e5f5fd0d13c3e06bfe9d5a422f40b5613c6a0f28f0a70e749b
Sep 12 00:16:47.247067 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.247038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f1aae7_285c_4898_be26_2e26dc23bcc2.slice/crio-090f66c51d510899c8debd8c15371e9035154a96058696d5eedfcb9e09ed570b WatchSource:0}: Error finding container 090f66c51d510899c8debd8c15371e9035154a96058696d5eedfcb9e09ed570b: Status 404 returned error can't find the container with id 090f66c51d510899c8debd8c15371e9035154a96058696d5eedfcb9e09ed570b
Sep 12 00:16:47.248160 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.248123    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecadca1_f699_49ea_9329_0c921a042679.slice/crio-4bd717ed3944360f75555d3ae1e49de7e5f1b53de4716ded0b4b9b39fe3491b8 WatchSource:0}: Error finding container 4bd717ed3944360f75555d3ae1e49de7e5f1b53de4716ded0b4b9b39fe3491b8: Status 404 returned error can't find the container with id 4bd717ed3944360f75555d3ae1e49de7e5f1b53de4716ded0b4b9b39fe3491b8
Sep 12 00:16:47.249398 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:16:47.249375    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7bca5f_401d_4c76_ac5f_305a5d176b80.slice/crio-0fdb1d0236a94e8d6240d95d41d4029e4a55cb85880ab28fe90932fcc508287c WatchSource:0}: Error finding container 0fdb1d0236a94e8d6240d95d41d4029e4a55cb85880ab28fe90932fcc508287c: Status 404 returned error can't find the container with id 0fdb1d0236a94e8d6240d95d41d4029e4a55cb85880ab28fe90932fcc508287c
Sep 12 00:16:47.268101 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.268074    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:11:46 +0000 UTC" deadline="2028-02-19 14:17:36.027347771 +0000 UTC"
Sep 12 00:16:47.268101 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.268099    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12614h0m48.759251672s"
Sep 12 00:16:47.368333 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.368304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:47.368483 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.368420    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:47.376209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.376174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxhxh" event={"ID":"52f1aae7-285c-4898-be26-2e26dc23bcc2","Type":"ContainerStarted","Data":"090f66c51d510899c8debd8c15371e9035154a96058696d5eedfcb9e09ed570b"}
Sep 12 00:16:47.378691 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.378659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jnf7d" event={"ID":"6e50127b-bff1-431e-aa20-3240d34fbc0b","Type":"ContainerStarted","Data":"fd5b4c6c1b364ec1bdbae07d720d0742e8c9f5a98209777e71e930fd1810a4dc"}
Sep 12 00:16:47.380671 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.380643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5vv4d" event={"ID":"21d75218-5ef8-477b-930e-0f3cac2806af","Type":"ContainerStarted","Data":"7036e40250e5afd4971a99361ad893350f081ddf9ce74dddc851cce2e1e7884f"}
Sep 12 00:16:47.381791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.381753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7lz5" event={"ID":"d4e849ab-ad3f-4296-b327-42951f5b82ab","Type":"ContainerStarted","Data":"118bf7ca8dacc8ed2e689d672e3fea51ec1bc76f118c6dc6b0e583ccc4cad903"}
Sep 12 00:16:47.382883 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.382858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59" event={"ID":"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c","Type":"ContainerStarted","Data":"f8e00c41aa5fe2e5f5fd0d13c3e06bfe9d5a422f40b5613c6a0f28f0a70e749b"}
Sep 12 00:16:47.384098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.384074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerStarted","Data":"446e09c732c0c153329f748742993b8e58e87ab93d31b86b29d99d60ebaaf542"}
Sep 12 00:16:47.385316 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.385297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"ede0ef9ed67d3ff9b43d28b6297be88e3a641ae5d8e53fe96a0d75a2834ef3fc"}
Sep 12 00:16:47.387341 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.387318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal" event={"ID":"9f84cc7b6dd936cd66de2a1a58b0376d","Type":"ContainerStarted","Data":"28f3c52cda977cbd3697db62f0e376a7bae65f80928bfa83525280e3744065ca"}
Sep 12 00:16:47.388539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.388515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f46p2" event={"ID":"7a7bca5f-401d-4c76-ac5f-305a5d176b80","Type":"ContainerStarted","Data":"0fdb1d0236a94e8d6240d95d41d4029e4a55cb85880ab28fe90932fcc508287c"}
Sep 12 00:16:47.389788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.389747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dlmjc" event={"ID":"1ecadca1-f699-49ea-9329-0c921a042679","Type":"ContainerStarted","Data":"4bd717ed3944360f75555d3ae1e49de7e5f1b53de4716ded0b4b9b39fe3491b8"}
Sep 12 00:16:47.403236 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.403191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-179.ec2.internal" podStartSLOduration=2.403178969 podStartE2EDuration="2.403178969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:47.403063466 +0000 UTC m=+3.537053690" watchObservedRunningTime="2026-09-12 00:16:47.403178969 +0000 UTC m=+3.537169193"
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.856540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.856605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:47.856654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.856792    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.856851    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:49.856832176 +0000 UTC m=+5.990822382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.856912    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.856945    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:16:49.85693352 +0000 UTC m=+5.990923727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.857021    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.857034    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.857046    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:47.857492 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:47.857081    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:49.85706966 +0000 UTC m=+5.991059872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:48.269099 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.269037    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:11:46 +0000 UTC" deadline="2028-05-21 03:09:50.159074158 +0000 UTC"
Sep 12 00:16:48.269099 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.269096    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14810h53m1.889981505s"
Sep 12 00:16:48.370648 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.370618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:48.370844 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:48.370752    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:48.371176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.371154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:48.371287 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:48.371266    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:48.400855 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.400816    2580 generic.go:356] "Generic (PLEG): container finished" podID="554e5250205817bcdbdb60ad48e7d590" containerID="664e945f15b8c59c73cb966700c11110e7c971dc75439494c30c22bd8d949d77" exitCode=0
Sep 12 00:16:48.401788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:48.401743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal" event={"ID":"554e5250205817bcdbdb60ad48e7d590","Type":"ContainerDied","Data":"664e945f15b8c59c73cb966700c11110e7c971dc75439494c30c22bd8d949d77"}
Sep 12 00:16:49.370992 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.368459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:49.370992 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.368586    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:49.427638 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.426907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal" event={"ID":"554e5250205817bcdbdb60ad48e7d590","Type":"ContainerStarted","Data":"aac2c81e7449633da59ff16d938fdfcdaa713fba5489142ce531da2c8c690370"}
Sep 12 00:16:49.442987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.442516    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-179.ec2.internal" podStartSLOduration=4.4424947 podStartE2EDuration="4.4424947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:49.442002015 +0000 UTC m=+5.575992241" watchObservedRunningTime="2026-09-12 00:16:49.4424947 +0000 UTC m=+5.576484927"
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.874849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.874906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:49.874953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875132    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875151    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875164    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875226    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:53.875207226 +0000 UTC m=+10.009197453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875626    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875682    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:53.875666755 +0000 UTC m=+10.009656960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875747    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:49.875829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:49.875794    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:16:53.875782964 +0000 UTC m=+10.009773172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:50.370067 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:50.370035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:50.370234 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:50.370169    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:50.370234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:50.370225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:50.370340 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:50.370292    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:51.368778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:51.368610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:51.368778 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:51.368748    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:52.369018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:52.368379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:52.369018 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:52.368579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:52.369639 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:52.369140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:52.369639 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:52.369263    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:53.368137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:53.367809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:53.368283 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.368203    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:53.914234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:53.914164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:53.914234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:53.914237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:53.914267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914395    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914406    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914425    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914440    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914453    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914463    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:17:01.914443672 +0000 UTC m=+18.048433888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914482    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:01.914471826 +0000 UTC m=+18.048462035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:53.914754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:53.914499    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:01.914490826 +0000 UTC m=+18.048481030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:54.368921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:54.368883    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:54.369105 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:54.368987    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:54.369105 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:54.369053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:54.369217 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:54.369118    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:55.368277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:55.367790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:55.368277 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:55.367928    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:56.368108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:56.368074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:56.368292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:56.368074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:56.368292 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:56.368201    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:56.368684 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:56.368291    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:57.368255 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:57.368223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:57.368430 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:57.368341    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:16:58.368077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:58.368042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:16:58.368296 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:58.368160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:16:58.368296 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:58.368216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:16:58.368407 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:58.368305    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:16:59.368434 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:16:59.368402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:16:59.368857 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:16:59.368532    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:00.368429 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:00.368395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:00.368582 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:00.368532    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:00.368978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:00.368586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:00.368978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:00.368698    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:01.368495 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:01.368462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:01.368690 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.368563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:01.969088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:01.969048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:01.969284 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:01.969099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:01.969284 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:01.969141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:01.969284 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969245    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:17:01.969284 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969265    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:17:01.969284 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969285    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:17:01.969497 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969297    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:17:01.969497 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969319    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:17.969302178 +0000 UTC m=+34.103292380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:17:01.969497 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969246    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:17:01.969497 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969347    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:17.969331406 +0000 UTC m=+34.103321609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:17:01.969497 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:01.969382    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:17:17.969371271 +0000 UTC m=+34.103361477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:17:02.367956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:02.367834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:02.368123 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:02.367969    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:02.368211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:02.367842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:02.368330 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:02.368293    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:03.368510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:03.368477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:03.368945 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:03.368606    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:04.369329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:04.369295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:04.369797 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:04.369385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:04.369797 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:04.369516    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:04.369797 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:04.369386    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:05.368092 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.367910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:05.370633 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:05.368516    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:05.463042 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.463009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dlmjc" event={"ID":"1ecadca1-f699-49ea-9329-0c921a042679","Type":"ContainerStarted","Data":"dd0190048e0ed839d6b433949bbde75b2e2f767cc5f38354e69788e258efa60e"}
Sep 12 00:17:05.464531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.464506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxhxh" event={"ID":"52f1aae7-285c-4898-be26-2e26dc23bcc2","Type":"ContainerStarted","Data":"a8b80d619cadc18145c1ae82bffba9c46251552b1c9c430e102c1e7de7ddd70f"}
Sep 12 00:17:05.466189 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.466145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jnf7d" event={"ID":"6e50127b-bff1-431e-aa20-3240d34fbc0b","Type":"ContainerStarted","Data":"31982cd8798cdaf6852a2d995fb1b641175a61baf8185e4fb09985d880ff2160"}
Sep 12 00:17:05.467513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.467485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5vv4d" event={"ID":"21d75218-5ef8-477b-930e-0f3cac2806af","Type":"ContainerStarted","Data":"c3c22ee184db3c123f265018213deeae54ed9e7f4ab7f22ba6f52d16b6b01a7e"}
Sep 12 00:17:05.469058 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.469033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7lz5" event={"ID":"d4e849ab-ad3f-4296-b327-42951f5b82ab","Type":"ContainerStarted","Data":"180911cd0cd31dcec19c095fcc45ed1b319e76c42f62ad9c09c3aea0a02d3dbd"}
Sep 12 00:17:05.470662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.470635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59" event={"ID":"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c","Type":"ContainerStarted","Data":"749baa3357c1cebb3e7ee7589ee1e58b307fc89ce30ab5abc2a3eded2b907200"}
Sep 12 00:17:05.472171 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.472149    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="81dcfff0c0f8a0378f3a0518801156489c9d4ba4baa45684a22ebf18dccf1766" exitCode=0
Sep 12 00:17:05.472386 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.472212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"81dcfff0c0f8a0378f3a0518801156489c9d4ba4baa45684a22ebf18dccf1766"}
Sep 12 00:17:05.474104 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.474039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"09dfab134515c9c1647d855e1e96918979ff181b8ef97a0babdc4711891b6d29"}
Sep 12 00:17:05.474170 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.474112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"60415e6497748c66935093e27017db277926da8ce15177f85349f99f9cc29c2c"}
Sep 12 00:17:05.479279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.479229    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dlmjc" podStartSLOduration=3.763028815 podStartE2EDuration="21.479213708s" totalImagesPullingTime="17.716184893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:05.479005125 +0000 UTC m=+21.612995350" watchObservedRunningTime="2026-09-12 00:17:05.479213708 +0000 UTC m=+21.613203935"
Sep 12 00:17:05.492360 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.492321    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5vv4d" podStartSLOduration=3.791965871 podStartE2EDuration="21.492304428s" totalImagesPullingTime="17.700338557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:05.491995859 +0000 UTC m=+21.625986095" watchObservedRunningTime="2026-09-12 00:17:05.492304428 +0000 UTC m=+21.626294653"
Sep 12 00:17:05.523188 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.523147    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jnf7d" podStartSLOduration=3.865988243 podStartE2EDuration="21.523132191s" totalImagesPullingTime="17.657143948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:05.522895397 +0000 UTC m=+21.656885622" watchObservedRunningTime="2026-09-12 00:17:05.523132191 +0000 UTC m=+21.657122448"
Sep 12 00:17:05.539170 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.539119    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t7lz5" podStartSLOduration=3.840524368 podStartE2EDuration="21.539101193s" totalImagesPullingTime="17.698576825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:05.539060415 +0000 UTC m=+21.673050636" watchObservedRunningTime="2026-09-12 00:17:05.539101193 +0000 UTC m=+21.673091420"
Sep 12 00:17:05.560014 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:05.559972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lxhxh" podStartSLOduration=3.905700565 podStartE2EDuration="21.559957771s" totalImagesPullingTime="17.654257206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:05.559859498 +0000 UTC m=+21.693849723" watchObservedRunningTime="2026-09-12 00:17:05.559957771 +0000 UTC m=+21.693947997"
Sep 12 00:17:06.368294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.368259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:06.368478 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:06.368368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:06.368478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.368423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:06.368595 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:06.368547    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:06.482875 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.482794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"6bc30f657b75dcfc6802928a1f1cff46bd496b6a0fe557c1615937668d5d2d92"}
Sep 12 00:17:06.483314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.482897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"adfffba786bf66046e975d1802d39aeb5bf86af44946604ec7a6e70bf2aca4bf"}
Sep 12 00:17:06.483314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.482937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"66cef87fce3717c7428c403ddcfe0e241cb1ff243bf11bad9b2c02c2984bbb25"}
Sep 12 00:17:06.483314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.482978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"18822e778278221c8a7b816240cf7441d047e563a98701f2aeec9cd4890c32b7"}
Sep 12 00:17:06.484804 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:06.484748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f46p2" event={"ID":"7a7bca5f-401d-4c76-ac5f-305a5d176b80","Type":"ContainerStarted","Data":"54146cbcb698b6f66a7fe312d9f46cae5044ff5cd13bc2c74c3d539d3ed0ea18"}
Sep 12 00:17:07.171137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.170877    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 12 00:17:07.292916 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.292690    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-12T00:17:07.171135555Z","UUID":"d98e0948-adf6-41d5-9f07-f319c47cee39","Handler":null,"Name":"","Endpoint":""}
Sep 12 00:17:07.294440 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.294414    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 12 00:17:07.294560 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.294451    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 12 00:17:07.368256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.368224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:07.368427 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:07.368355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:07.489399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:07.489365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59" event={"ID":"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c","Type":"ContainerStarted","Data":"15df43af88bbd3864c670bb51e23d9cb98ef4699f0f2cd90be09e509048eb10e"}
Sep 12 00:17:08.367904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:08.367863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:08.368068 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:08.367863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:08.368068 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:08.367999    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:08.368190 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:08.368103    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:08.495514 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:08.495480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"ec334da34f81646afd09706702b9e550ebdd92df5789ddc6dd7bf8ebcada5e47"}
Sep 12 00:17:09.367918 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:09.367888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:09.368122 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:09.368004    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:10.298823 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.298562    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:17:10.299307 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.299085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:17:10.313989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.313949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f46p2" podStartSLOduration=8.661718985 podStartE2EDuration="26.313933398s" totalImagesPullingTime="17.652214413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:06.498148415 +0000 UTC m=+22.632138640" watchObservedRunningTime="2026-09-12 00:17:10.313933398 +0000 UTC m=+26.447923620"
Sep 12 00:17:10.368482 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.368409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:10.368482 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.368409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:10.368687 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:10.368524    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:10.368744 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:10.368706    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:10.501780 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.501738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59" event={"ID":"1c33dd74-8ecb-4929-b91c-4d95bda4aa8c","Type":"ContainerStarted","Data":"0e949d5115de09db1a4450d2a4441abefe890da84149452c5df0408fd797ca73"}
Sep 12 00:17:10.503548 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.503522    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="d8df0514aabf950541e1406d550c665b353563de7ba585d411b780f7ffa5375b" exitCode=0
Sep 12 00:17:10.503663 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.503604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"d8df0514aabf950541e1406d550c665b353563de7ba585d411b780f7ffa5375b"}
Sep 12 00:17:10.507475 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.507457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" event={"ID":"7385b79c-9e06-4e15-a915-d043b242415c","Type":"ContainerStarted","Data":"13fcb841c8306e1696ac56c016257ca8095bf93398ef1cb266f101082a9f53b3"}
Sep 12 00:17:10.507690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.507674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:17:10.508113 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.508099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lxhxh"
Sep 12 00:17:10.521888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.521850    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdq59" podStartSLOduration=4.28099568 podStartE2EDuration="26.521837294s" totalImagesPullingTime="22.240841614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:10.521410602 +0000 UTC m=+26.655400826" watchObservedRunningTime="2026-09-12 00:17:10.521837294 +0000 UTC m=+26.655827518"
Sep 12 00:17:10.582071 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:10.582021    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5" podStartSLOduration=8.667671909 podStartE2EDuration="26.582006637s" totalImagesPullingTime="17.914334728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:10.580705042 +0000 UTC m=+26.714695279" watchObservedRunningTime="2026-09-12 00:17:10.582006637 +0000 UTC m=+26.715996873"
Sep 12 00:17:11.368634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.368601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:11.369396 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:11.368732    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:11.512141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.512106    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="e32c97d5398129846034f9912868dc305aadc615f324dd4ddf501df4376698fb" exitCode=0
Sep 12 00:17:11.512314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.512270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"e32c97d5398129846034f9912868dc305aadc615f324dd4ddf501df4376698fb"}
Sep 12 00:17:11.512314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.512306    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 12 00:17:11.512717 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.512693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:11.512877 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.512806    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:11.529397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.529376    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:11.530361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:11.530338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:12.050341 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.050163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6ztlj"]
Sep 12 00:17:12.050557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.050450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:12.050626 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:12.050561    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:12.052918 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.052888    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72whv"]
Sep 12 00:17:12.053028 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.052989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:12.053087 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:12.053063    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:12.053481 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.053460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhbcl"]
Sep 12 00:17:12.053568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.053556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:12.053664 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:12.053640    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:12.517257 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.517167    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="19702364fc336b1274ad8202e734cf1cadb713d2f4f27cd6ab31f0e0c140c1e2" exitCode=0
Sep 12 00:17:12.517809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.517252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"19702364fc336b1274ad8202e734cf1cadb713d2f4f27cd6ab31f0e0c140c1e2"}
Sep 12 00:17:12.517809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.517391    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 12 00:17:12.570073 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:12.570042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:13.368004 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:13.367928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:13.368004 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:13.367984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:13.368201 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:13.368043    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:13.368201 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:13.368088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:13.368201 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:13.368163    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:13.368319 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:13.368273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:15.368055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:15.368017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:15.368055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:15.368042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:15.368055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:15.368062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:15.368574 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:15.368259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:15.368574 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:15.368271    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:15.368574 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:15.368365    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:17.368043 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.368006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:17.368503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.368052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:17.368503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.368059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:17.368503 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.368131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72whv" podUID="f79f9dca-61f6-45f2-b1e5-248e521d599f"
Sep 12 00:17:17.368503 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.368193    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ztlj" podUID="5989bb1c-1955-454c-8a17-8b1b698ead67"
Sep 12 00:17:17.368503 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.368272    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:17:17.531828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.531739    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="0bd04647412bdc01323d743161a10c9b3dacd7e2839ad5d819a84534b6126a77" exitCode=0
Sep 12 00:17:17.531828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.531797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"0bd04647412bdc01323d743161a10c9b3dacd7e2839ad5d819a84534b6126a77"}
Sep 12 00:17:17.668003 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.667831    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-179.ec2.internal" event="NodeReady"
Sep 12 00:17:17.668125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.668113    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 12 00:17:17.708440 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.708410    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xx595"]
Sep 12 00:17:17.724285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.724250    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wrrp5"]
Sep 12 00:17:17.724451 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.724368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.726325 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.726298    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 12 00:17:17.726524 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.726510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 12 00:17:17.726632 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.726535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knhwr\""
Sep 12 00:17:17.730342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.730323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx595"]
Sep 12 00:17:17.730450 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.730354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrrp5"]
Sep 12 00:17:17.730450 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.730438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.732445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.732424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ns4k9\""
Sep 12 00:17:17.732556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.732463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 12 00:17:17.732556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.732428    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 12 00:17:17.732556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.732433    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 12 00:17:17.884799 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-tmp-dir\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.884799 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b3ab479-faf4-46d8-af56-b8295bb9c41d-cert\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.884991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pw4w\" (UniqueName: \"kubernetes.io/projected/7b3ab479-faf4-46d8-af56-b8295bb9c41d-kube-api-access-4pw4w\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.884991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-config-volume\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.884991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6bst\" (UniqueName: \"kubernetes.io/projected/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-kube-api-access-x6bst\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.884991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.884940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986016 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.985981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:17.986016 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pw4w\" (UniqueName: \"kubernetes.io/projected/7b3ab479-faf4-46d8-af56-b8295bb9c41d-kube-api-access-4pw4w\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-config-volume\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6bst\" (UniqueName: \"kubernetes.io/projected/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-kube-api-access-x6bst\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-tmp-dir\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986110    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b3ab479-faf4-46d8-af56-b8295bb9c41d-cert\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret podName:f79f9dca-61f6-45f2-b1e5-248e521d599f nodeName:}" failed. No retries permitted until 2026-09-12 00:17:49.986164601 +0000 UTC m=+66.120154804 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret") pod "global-pull-secret-syncer-72whv" (UID: "f79f9dca-61f6-45f2-b1e5-248e521d599f") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986216    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986250    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986269    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:17:17.986268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986273    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:18.486255208 +0000 UTC m=+34.620245416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986281    2580 projected.go:196] Error preparing data for projected volume kube-api-access-mprv4 for pod openshift-network-diagnostics/network-check-target-6ztlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986308    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986324    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4 podName:5989bb1c-1955-454c-8a17-8b1b698ead67 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:49.98631228 +0000 UTC m=+66.120302487 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mprv4" (UniqueName: "kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4") pod "network-check-target-6ztlj" (UID: "5989bb1c-1955-454c-8a17-8b1b698ead67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:17.986345    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:49.986334324 +0000 UTC m=+66.120324558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-tmp-dir\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.986828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.986824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-config-volume\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.990420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.990398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b3ab479-faf4-46d8-af56-b8295bb9c41d-cert\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:17.993910 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.993887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6bst\" (UniqueName: \"kubernetes.io/projected/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-kube-api-access-x6bst\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:17.994127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:17.994110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pw4w\" (UniqueName: \"kubernetes.io/projected/7b3ab479-faf4-46d8-af56-b8295bb9c41d-kube-api-access-4pw4w\") pod \"ingress-canary-wrrp5\" (UID: \"7b3ab479-faf4-46d8-af56-b8295bb9c41d\") " pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:18.069391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.069353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrrp5"
Sep 12 00:17:18.393731 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.393697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrrp5"]
Sep 12 00:17:18.397186 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:17:18.397159    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3ab479_faf4_46d8_af56_b8295bb9c41d.slice/crio-c5ad89b7ee16ffe2ee0d89de513166d886a087488d79c856bbb54e2f747eb106 WatchSource:0}: Error finding container c5ad89b7ee16ffe2ee0d89de513166d886a087488d79c856bbb54e2f747eb106: Status 404 returned error can't find the container with id c5ad89b7ee16ffe2ee0d89de513166d886a087488d79c856bbb54e2f747eb106
Sep 12 00:17:18.490180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.490092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:18.490325 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:18.490267    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:18.490369 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:18.490337    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:19.490320378 +0000 UTC m=+35.624310581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:18.537081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.537037    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe313196-f058-4a7a-9de7-6307e39a7d4a" containerID="433dcef331f1b04de1c9421093cfbe6344d419ca6db9b2f2ebce8e6c1debb5ce" exitCode=0
Sep 12 00:17:18.537257 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.537119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerDied","Data":"433dcef331f1b04de1c9421093cfbe6344d419ca6db9b2f2ebce8e6c1debb5ce"}
Sep 12 00:17:18.538294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:18.538270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrrp5" event={"ID":"7b3ab479-faf4-46d8-af56-b8295bb9c41d","Type":"ContainerStarted","Data":"c5ad89b7ee16ffe2ee0d89de513166d886a087488d79c856bbb54e2f747eb106"}
Sep 12 00:17:19.368595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.368561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:19.368595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.368594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:19.368861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.368596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:19.371993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.371784    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 00:17:19.371993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.371798    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 00:17:19.371993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.371816    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 00:17:19.371993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.371856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vv7j9\""
Sep 12 00:17:19.371993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.371801    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 00:17:19.372331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.372185    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tdrkx\""
Sep 12 00:17:19.498381 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.498349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:19.498927 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:19.498517    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:19.498927 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:19.498590    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:21.498568853 +0000 UTC m=+37.632559063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:19.545640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.545607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" event={"ID":"fe313196-f058-4a7a-9de7-6307e39a7d4a","Type":"ContainerStarted","Data":"89bbc99fce09fa3f64b1cfba37721a240a2d1285f118556e32094234f9051acd"}
Sep 12 00:17:19.564213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:19.564155    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vmbg5" podStartSLOduration=7.436739344 podStartE2EDuration="35.564138138s" totalImagesPullingTime="28.127398794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:19.563098196 +0000 UTC m=+35.697088424" watchObservedRunningTime="2026-09-12 00:17:19.564138138 +0000 UTC m=+35.698128362"
Sep 12 00:17:21.514875 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:21.514832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:21.515349 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:21.514941    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:21.515349 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:21.515026    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:25.515002272 +0000 UTC m=+41.648992489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:21.551095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:21.551065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrrp5" event={"ID":"7b3ab479-faf4-46d8-af56-b8295bb9c41d","Type":"ContainerStarted","Data":"e9f73533f6dd2aedb924dfc0bd0e4ca7c6aac62d835278df83e84f4ef3e4fe24"}
Sep 12 00:17:21.565713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:21.565669    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wrrp5" podStartSLOduration=2.08195459 podStartE2EDuration="4.565654328s" totalImagesPullingTime="2.483699738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:21.564405854 +0000 UTC m=+37.698396078" watchObservedRunningTime="2026-09-12 00:17:21.565654328 +0000 UTC m=+37.699644553"
Sep 12 00:17:25.540967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:25.540929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:25.541379 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:25.541087    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:25.541379 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:25.541153    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:33.541137201 +0000 UTC m=+49.675127403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:33.596042 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:33.595875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:33.596682 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:33.596601    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:33.597521 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:33.597497    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:49.5974449 +0000 UTC m=+65.731435117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:44.531530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:44.531502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-42sn5"
Sep 12 00:17:49.606994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:49.606957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:17:49.607387 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:49.607076    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:49.607387 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:49.607129    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:18:21.607114619 +0000 UTC m=+97.741104822 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:50.008490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.008441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:17:50.008490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.008494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:50.008716 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.008532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:50.011067 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.011045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 00:17:50.011067 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.011055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 00:17:50.011203 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.011055    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 00:17:50.018847 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:50.018828    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:17:50.018913 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:17:50.018885    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:18:54.018869534 +0000 UTC m=+130.152859737 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : secret "metrics-daemon-secret" not found
Sep 12 00:17:50.020519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.020502    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 00:17:50.022041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.022020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f79f9dca-61f6-45f2-b1e5-248e521d599f-original-pull-secret\") pod \"global-pull-secret-syncer-72whv\" (UID: \"f79f9dca-61f6-45f2-b1e5-248e521d599f\") " pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:50.031187 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.031166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprv4\" (UniqueName: \"kubernetes.io/projected/5989bb1c-1955-454c-8a17-8b1b698ead67-kube-api-access-mprv4\") pod \"network-check-target-6ztlj\" (UID: \"5989bb1c-1955-454c-8a17-8b1b698ead67\") " pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:50.280958 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.280870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72whv"
Sep 12 00:17:50.291068 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.291043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tdrkx\""
Sep 12 00:17:50.299150 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.299129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:50.549049 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.548978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72whv"]
Sep 12 00:17:50.553217 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:17:50.553184    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79f9dca_61f6_45f2_b1e5_248e521d599f.slice/crio-b79cb5114e3f8814e6e75045ab923126778497e8b42cd7ca39e6aba27e0cb7b1 WatchSource:0}: Error finding container b79cb5114e3f8814e6e75045ab923126778497e8b42cd7ca39e6aba27e0cb7b1: Status 404 returned error can't find the container with id b79cb5114e3f8814e6e75045ab923126778497e8b42cd7ca39e6aba27e0cb7b1
Sep 12 00:17:50.569749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.569721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6ztlj"]
Sep 12 00:17:50.574067 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:17:50.574040    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5989bb1c_1955_454c_8a17_8b1b698ead67.slice/crio-b227f65863ae8fcd6fee25c2858673478c2b8c5ff24fa8aba0b02726bd5cabc9 WatchSource:0}: Error finding container b227f65863ae8fcd6fee25c2858673478c2b8c5ff24fa8aba0b02726bd5cabc9: Status 404 returned error can't find the container with id b227f65863ae8fcd6fee25c2858673478c2b8c5ff24fa8aba0b02726bd5cabc9
Sep 12 00:17:50.623659 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.623622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72whv" event={"ID":"f79f9dca-61f6-45f2-b1e5-248e521d599f","Type":"ContainerStarted","Data":"b79cb5114e3f8814e6e75045ab923126778497e8b42cd7ca39e6aba27e0cb7b1"}
Sep 12 00:17:50.624778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:50.624746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6ztlj" event={"ID":"5989bb1c-1955-454c-8a17-8b1b698ead67","Type":"ContainerStarted","Data":"b227f65863ae8fcd6fee25c2858673478c2b8c5ff24fa8aba0b02726bd5cabc9"}
Sep 12 00:17:55.639869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:55.639756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6ztlj" event={"ID":"5989bb1c-1955-454c-8a17-8b1b698ead67","Type":"ContainerStarted","Data":"32c9227aac427a2b51366a88c2c56b11da06b487111444ec1962f6ed8a1504d7"}
Sep 12 00:17:55.640327 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:55.639889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:17:55.641344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:55.641321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72whv" event={"ID":"f79f9dca-61f6-45f2-b1e5-248e521d599f","Type":"ContainerStarted","Data":"2699661b9fb2e21b556f94ed1e66ffef9867f19ceb565961fe1b54058bc29781"}
Sep 12 00:17:55.654946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:55.654900    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6ztlj" podStartSLOduration=66.880240458 podStartE2EDuration="1m11.654888079s" totalImagesPullingTime="4.774647621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:55.65382556 +0000 UTC m=+71.787815785" watchObservedRunningTime="2026-09-12 00:17:55.654888079 +0000 UTC m=+71.788878336"
Sep 12 00:17:55.669005 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:17:55.668962    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-72whv" podStartSLOduration=66.868410226 podStartE2EDuration="1m11.668949104s" totalImagesPullingTime="4.800538878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:55.668332362 +0000 UTC m=+71.802322586" watchObservedRunningTime="2026-09-12 00:17:55.668949104 +0000 UTC m=+71.802939332"
Sep 12 00:18:08.750657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.750538    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"]
Sep 12 00:18:08.754776 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.754746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.757163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.757139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 12 00:18:08.757163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.757157    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 12 00:18:08.758092 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.758069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 12 00:18:08.758092 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.758082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 12 00:18:08.758259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.758095    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 12 00:18:08.758259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.758080    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 12 00:18:08.758259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.758170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 12 00:18:08.764345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.764325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"]
Sep 12 00:18:08.828581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.828735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66xhs\" (UniqueName: \"kubernetes.io/projected/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-kube-api-access-66xhs\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.828735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.828735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.828897 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.828897 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.828790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929604 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66xhs\" (UniqueName: \"kubernetes.io/projected/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-kube-api-access-66xhs\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.929944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.929840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.930447 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.930416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.932180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.932151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-ca\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.932296 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.932199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.932356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.932306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.932356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.932341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:08.937551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:08.937528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66xhs\" (UniqueName: \"kubernetes.io/projected/dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29-kube-api-access-66xhs\") pod \"cluster-proxy-proxy-agent-bf5f696b6-2pdxz\" (UID: \"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:09.073904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:09.073821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:18:09.347211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:09.347139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"]
Sep 12 00:18:09.350108 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:18:09.350076    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc90d4d5_a0d1_46fe_9f46_e7c5a4f2be29.slice/crio-7ca4c1aca16e918acaea0327c1f195933322ef7c906b6890956e99541118e7a8 WatchSource:0}: Error finding container 7ca4c1aca16e918acaea0327c1f195933322ef7c906b6890956e99541118e7a8: Status 404 returned error can't find the container with id 7ca4c1aca16e918acaea0327c1f195933322ef7c906b6890956e99541118e7a8
Sep 12 00:18:09.677454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:09.677420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerStarted","Data":"7ca4c1aca16e918acaea0327c1f195933322ef7c906b6890956e99541118e7a8"}
Sep 12 00:18:13.690727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:13.690690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerStarted","Data":"6117f69070169e8346c9be56e1f7ebb2b405fe69b278da0a7cf2d4bf03b51222"}
Sep 12 00:18:13.690727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:13.690735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerStarted","Data":"fb2d513d07a2511b9ec4078e8331de19676681060a7fa7e7bae7d4384fda15d9"}
Sep 12 00:18:13.691131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:13.690748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerStarted","Data":"d9b0ff8e8497f030e9bb655b3c1c9416514c63a4f4d7b737b08d2c7a500e0dbf"}
Sep 12 00:18:13.707749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:13.707708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" podStartSLOduration=1.998902075 podStartE2EDuration="5.707694442s" totalImagesPullingTime="3.708792367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:18:13.707347243 +0000 UTC m=+89.841337467" watchObservedRunningTime="2026-09-12 00:18:13.707694442 +0000 UTC m=+89.841684666"
Sep 12 00:18:21.613719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:21.613687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:18:21.614067 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:18:21.613841    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:18:21.614067 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:18:21.613919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:19:25.613903533 +0000 UTC m=+161.747893736 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:18:26.646282 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:26.646250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6ztlj"
Sep 12 00:18:54.035466 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:18:54.035427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:18:54.035978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:18:54.035547    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:18:54.035978 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:18:54.035601    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs podName:589333b5-36cc-4fc7-99a3-7ab405c26767 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:56.035586966 +0000 UTC m=+252.169577169 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs") pod "network-metrics-daemon-jhbcl" (UID: "589333b5-36cc-4fc7-99a3-7ab405c26767") : secret "metrics-daemon-secret" not found
Sep 12 00:19:20.764364 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:19:20.764313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xx595" podUID="3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6"
Sep 12 00:19:20.877436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:19:20.877409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx595"
Sep 12 00:19:22.394478 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:19:22.394432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jhbcl" podUID="589333b5-36cc-4fc7-99a3-7ab405c26767"
Sep 12 00:19:25.649329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:19:25.649277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:19:25.649738 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:19:25.649405    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:19:25.649738 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:19:25.649464    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls podName:3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6 nodeName:}" failed. No retries permitted until 2026-09-12 00:21:27.649447053 +0000 UTC m=+283.783437259 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls") pod "dns-default-xx595" (UID: "3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6") : secret "dns-default-metrics-tls" not found
Sep 12 00:19:35.368141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:19:35.368101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:20:29.846146 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.846114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"]
Sep 12 00:20:29.847850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.847834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.851047 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.851004    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 12 00:20:29.851867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.851847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 12 00:20:29.851983 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.851946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 12 00:20:29.852035 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.852012    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r8l76\""
Sep 12 00:20:29.852085 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.852062    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 12 00:20:29.860834 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.860815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"]
Sep 12 00:20:29.888628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.888599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.888758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.888654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6019a323-7283-4391-916d-10bf862e0cff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.888758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.888681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2sjv\" (UniqueName: \"kubernetes.io/projected/6019a323-7283-4391-916d-10bf862e0cff-kube-api-access-x2sjv\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.989155 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.989116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2sjv\" (UniqueName: \"kubernetes.io/projected/6019a323-7283-4391-916d-10bf862e0cff-kube-api-access-x2sjv\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.989356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.989176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.989356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.989235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6019a323-7283-4391-916d-10bf862e0cff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.989356 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:29.989329    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:29.989525 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:29.989403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls podName:6019a323-7283-4391-916d-10bf862e0cff nodeName:}" failed. No retries permitted until 2026-09-12 00:20:30.489388194 +0000 UTC m=+226.623378397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-55xws" (UID: "6019a323-7283-4391-916d-10bf862e0cff") : secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:29.989984 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.989962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6019a323-7283-4391-916d-10bf862e0cff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:29.996791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:29.996758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2sjv\" (UniqueName: \"kubernetes.io/projected/6019a323-7283-4391-916d-10bf862e0cff-kube-api-access-x2sjv\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:30.493031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:30.492988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:30.493219 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:30.493107    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:30.493219 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:30.493163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls podName:6019a323-7283-4391-916d-10bf862e0cff nodeName:}" failed. No retries permitted until 2026-09-12 00:20:31.49315003 +0000 UTC m=+227.627140233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-55xws" (UID: "6019a323-7283-4391-916d-10bf862e0cff") : secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:31.499636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:31.499597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:31.500026 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:31.499695    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:31.500026 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:31.499745    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls podName:6019a323-7283-4391-916d-10bf862e0cff nodeName:}" failed. No retries permitted until 2026-09-12 00:20:33.49973231 +0000 UTC m=+229.633722512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-55xws" (UID: "6019a323-7283-4391-916d-10bf862e0cff") : secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:33.515664 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:33.515619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:33.516078 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:33.515806    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:33.516078 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:33.515908    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls podName:6019a323-7283-4391-916d-10bf862e0cff nodeName:}" failed. No retries permitted until 2026-09-12 00:20:37.515889179 +0000 UTC m=+233.649879382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-55xws" (UID: "6019a323-7283-4391-916d-10bf862e0cff") : secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:35.006298 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.006260    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-svbxq"]
Sep 12 00:20:35.009251 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.009231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.011575 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.011551    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 12 00:20:35.011679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.011562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 12 00:20:35.011679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.011618    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pwzwl\""
Sep 12 00:20:35.011679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.011588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 12 00:20:35.012242 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.012225    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 12 00:20:35.019086 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.019061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-svbxq"]
Sep 12 00:20:35.127778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.127738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.127949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.127859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcf1f4a7-b086-434a-97a4-a03af21aef51-data-volume\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.127949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.127893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.127949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.127912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zxhq\" (UniqueName: \"kubernetes.io/projected/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-api-access-6zxhq\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.127949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.127937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcf1f4a7-b086-434a-97a4-a03af21aef51-crio-socket\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228475 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228565 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcf1f4a7-b086-434a-97a4-a03af21aef51-data-volume\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228613 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.228588    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.228613 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zxhq\" (UniqueName: \"kubernetes.io/projected/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-api-access-6zxhq\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228690 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.228664    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:35.728644147 +0000 UTC m=+231.862634356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.228819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcf1f4a7-b086-434a-97a4-a03af21aef51-crio-socket\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228819 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.228742    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.228907 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcf1f4a7-b086-434a-97a4-a03af21aef51-data-volume\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.228907 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.228838    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:35.728819965 +0000 UTC m=+231.862810199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.228907 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.228838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcf1f4a7-b086-434a-97a4-a03af21aef51-crio-socket\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.236095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.236073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zxhq\" (UniqueName: \"kubernetes.io/projected/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-api-access-6zxhq\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.733807 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.733756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.733972 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.733904    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.733972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:35.733926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:35.733972 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.733964    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:36.733948927 +0000 UTC m=+232.867939135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.734132 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.733991    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.734132 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:35.734043    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:36.734030956 +0000 UTC m=+232.868021159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:36.740657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:36.740623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:36.741080 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:36.740704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:36.741080 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:36.740797    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:36.741080 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:36.740838    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:36.741080 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:36.740884    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:38.740859418 +0000 UTC m=+234.874849689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:36.741080 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:36.740899    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:38.740892999 +0000 UTC m=+234.874883205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:36.777900 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:36.777874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5vv4d_21d75218-5ef8-477b-930e-0f3cac2806af/dns-node-resolver/0.log"
Sep 12 00:20:37.544866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:37.544812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:37.545047 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:37.544962    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:37.545047 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:37.545023    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls podName:6019a323-7283-4391-916d-10bf862e0cff nodeName:}" failed. No retries permitted until 2026-09-12 00:20:45.54500738 +0000 UTC m=+241.678997597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-55xws" (UID: "6019a323-7283-4391-916d-10bf862e0cff") : secret "cluster-monitoring-operator-tls" not found
Sep 12 00:20:37.977709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:37.977682    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jnf7d_6e50127b-bff1-431e-aa20-3240d34fbc0b/node-ca/0.log"
Sep 12 00:20:38.754931 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:38.754889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:38.755146 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:38.754996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:38.755146 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:38.755013    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:38.755146 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:38.755079    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:42.755063388 +0000 UTC m=+238.889053592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:38.755146 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:38.755105    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:38.755313 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:38.755158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:42.755142591 +0000 UTC m=+238.889132794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:38.977801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:38.977760    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrrp5_7b3ab479-faf4-46d8-af56-b8295bb9c41d/serve-healthcheck-canary/0.log"
Sep 12 00:20:42.787044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:42.787007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:42.787424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:42.787061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:42.787424 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:42.787136    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:42.787424 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:42.787159    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:42.787424 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:42.787194    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:50.787180902 +0000 UTC m=+246.921171105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:42.787424 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:20:42.787208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm podName:dcf1f4a7-b086-434a-97a4-a03af21aef51 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:50.787201707 +0000 UTC m=+246.921191910 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-svbxq" (UID: "dcf1f4a7-b086-434a-97a4-a03af21aef51") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:45.607478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:45.607434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:45.609818 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:45.609786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6019a323-7283-4391-916d-10bf862e0cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-55xws\" (UID: \"6019a323-7283-4391-916d-10bf862e0cff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:45.758810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:45.758781    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r8l76\""
Sep 12 00:20:45.768025 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:45.768003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"
Sep 12 00:20:46.032578 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:46.031628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws"]
Sep 12 00:20:46.038631 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:20:46.038603    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6019a323_7283_4391_916d_10bf862e0cff.slice/crio-9f193797fbc0f18b7ac684cd5e610d778e99d36e81ae4e97746d053cf8930872 WatchSource:0}: Error finding container 9f193797fbc0f18b7ac684cd5e610d778e99d36e81ae4e97746d053cf8930872: Status 404 returned error can't find the container with id 9f193797fbc0f18b7ac684cd5e610d778e99d36e81ae4e97746d053cf8930872
Sep 12 00:20:46.110280 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:46.110242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws" event={"ID":"6019a323-7283-4391-916d-10bf862e0cff","Type":"ContainerStarted","Data":"9f193797fbc0f18b7ac684cd5e610d778e99d36e81ae4e97746d053cf8930872"}
Sep 12 00:20:48.118880 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:48.118781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws" event={"ID":"6019a323-7283-4391-916d-10bf862e0cff","Type":"ContainerStarted","Data":"426307a8ebc34eeb977c27af537c8dff853df7dde36ee2e1c007bbbd41ef8774"}
Sep 12 00:20:48.133288 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:48.133227    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-55xws" podStartSLOduration=17.412316132 podStartE2EDuration="19.133210437s" totalImagesPullingTime="1.720894305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:48.132256336 +0000 UTC m=+244.266246561" watchObservedRunningTime="2026-09-12 00:20:48.133210437 +0000 UTC m=+244.267200662"
Sep 12 00:20:49.075211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:49.075170    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" podUID="dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 12 00:20:50.848105 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.848052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:50.848105 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.848122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:50.848560 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.848542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcf1f4a7-b086-434a-97a4-a03af21aef51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:50.850275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.850255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcf1f4a7-b086-434a-97a4-a03af21aef51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-svbxq\" (UID: \"dcf1f4a7-b086-434a-97a4-a03af21aef51\") " pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:50.921295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.921264    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pwzwl\""
Sep 12 00:20:50.930041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:50.930021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-svbxq"
Sep 12 00:20:51.193653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:51.193620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-svbxq"]
Sep 12 00:20:51.196958 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:20:51.196928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf1f4a7_b086_434a_97a4_a03af21aef51.slice/crio-1a878cc9721514e5f251f0e12b03acbabc38190112a5573f03d42ce23f5c0956 WatchSource:0}: Error finding container 1a878cc9721514e5f251f0e12b03acbabc38190112a5573f03d42ce23f5c0956: Status 404 returned error can't find the container with id 1a878cc9721514e5f251f0e12b03acbabc38190112a5573f03d42ce23f5c0956
Sep 12 00:20:52.131478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:52.131446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-svbxq" event={"ID":"dcf1f4a7-b086-434a-97a4-a03af21aef51","Type":"ContainerStarted","Data":"8a3626147f01053e7673bda7c684d7f01574a55e5ebf43c6592eb8ba2b2d9deb"}
Sep 12 00:20:52.131827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:52.131498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-svbxq" event={"ID":"dcf1f4a7-b086-434a-97a4-a03af21aef51","Type":"ContainerStarted","Data":"44fba10236e728061c14bc0ed0dd62f359edb5eac66b47d1295f856391735e5b"}
Sep 12 00:20:52.131827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:52.131516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-svbxq" event={"ID":"dcf1f4a7-b086-434a-97a4-a03af21aef51","Type":"ContainerStarted","Data":"1a878cc9721514e5f251f0e12b03acbabc38190112a5573f03d42ce23f5c0956"}
Sep 12 00:20:54.139281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.139245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-svbxq" event={"ID":"dcf1f4a7-b086-434a-97a4-a03af21aef51","Type":"ContainerStarted","Data":"c2d6648185a1c76249e69c3e3f6b5b49217e10787fc77d68413ce103b9f6d7da"}
Sep 12 00:20:54.155460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.155417    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-svbxq" podStartSLOduration=18.11255849 podStartE2EDuration="20.155403784s" totalImagesPullingTime="2.042845294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:20:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:54.154339616 +0000 UTC m=+250.288329834" watchObservedRunningTime="2026-09-12 00:20:54.155403784 +0000 UTC m=+250.289394009"
Sep 12 00:20:54.731203 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.731170    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"]
Sep 12 00:20:54.733399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.733375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:54.737198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.737179    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-plxd8\""
Sep 12 00:20:54.737288 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.737207    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 12 00:20:54.749659 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.749636    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"]
Sep 12 00:20:54.879990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.879950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-knhxf\" (UID: \"7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:54.980393 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.980357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-knhxf\" (UID: \"7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:54.982850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:54.982797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-knhxf\" (UID: \"7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:55.041541 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:55.041509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:55.306047 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:55.305958    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"]
Sep 12 00:20:55.310905 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:20:55.310883    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1cdeb9_aaf7_4d47_8ff7_7d742a72cfe6.slice/crio-2eecc3ac4891313e44072663d12f000fa3e158ff457fafec172debd4dd777bd5 WatchSource:0}: Error finding container 2eecc3ac4891313e44072663d12f000fa3e158ff457fafec172debd4dd777bd5: Status 404 returned error can't find the container with id 2eecc3ac4891313e44072663d12f000fa3e158ff457fafec172debd4dd777bd5
Sep 12 00:20:56.088204 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.088167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:20:56.090370 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.090350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/589333b5-36cc-4fc7-99a3-7ab405c26767-metrics-certs\") pod \"network-metrics-daemon-jhbcl\" (UID: \"589333b5-36cc-4fc7-99a3-7ab405c26767\") " pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:20:56.146375 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.146336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf" event={"ID":"7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6","Type":"ContainerStarted","Data":"2eecc3ac4891313e44072663d12f000fa3e158ff457fafec172debd4dd777bd5"}
Sep 12 00:20:56.371345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.371266    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vv7j9\""
Sep 12 00:20:56.380025 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.379996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhbcl"
Sep 12 00:20:56.701749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:56.701711    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhbcl"]
Sep 12 00:20:56.706551 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:20:56.706522    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589333b5_36cc_4fc7_99a3_7ab405c26767.slice/crio-5785b6f2c6bbdfdc5bfabe31dc39d3ddcc7d64cb3b027647659ba8c3a34f7b31 WatchSource:0}: Error finding container 5785b6f2c6bbdfdc5bfabe31dc39d3ddcc7d64cb3b027647659ba8c3a34f7b31: Status 404 returned error can't find the container with id 5785b6f2c6bbdfdc5bfabe31dc39d3ddcc7d64cb3b027647659ba8c3a34f7b31
Sep 12 00:20:57.150541 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.150455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf" event={"ID":"7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6","Type":"ContainerStarted","Data":"2403dbdb91b0d7f017ab08a179c75ae8f0b837b0239914cb26a71740157237ff"}
Sep 12 00:20:57.150700 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.150636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:57.151712 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.151689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhbcl" event={"ID":"589333b5-36cc-4fc7-99a3-7ab405c26767","Type":"ContainerStarted","Data":"5785b6f2c6bbdfdc5bfabe31dc39d3ddcc7d64cb3b027647659ba8c3a34f7b31"}
Sep 12 00:20:57.155399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.155378    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf"
Sep 12 00:20:57.165196 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.165159    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-knhxf" podStartSLOduration=2.017017053 podStartE2EDuration="3.165145533s" totalImagesPullingTime="1.14812848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:57.163743406 +0000 UTC m=+253.297733630" watchObservedRunningTime="2026-09-12 00:20:57.165145533 +0000 UTC m=+253.299135758"
Sep 12 00:20:57.288233 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.288201    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"]
Sep 12 00:20:57.290153 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.290132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.292163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.292134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 12 00:20:57.292674 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.292326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 12 00:20:57.292674 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.292410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8k6dh\""
Sep 12 00:20:57.292674 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.292539    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 12 00:20:57.299707 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.299680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"]
Sep 12 00:20:57.397739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.397699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72d6568b-7488-499f-b4be-e177b12de27c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.398169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.397745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.398169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.397834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krks2\" (UniqueName: \"kubernetes.io/projected/72d6568b-7488-499f-b4be-e177b12de27c-kube-api-access-krks2\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.398169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.397865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.499122 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.499079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krks2\" (UniqueName: \"kubernetes.io/projected/72d6568b-7488-499f-b4be-e177b12de27c-kube-api-access-krks2\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.499290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.499139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.499290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.499197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72d6568b-7488-499f-b4be-e177b12de27c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.499290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.499230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.500013 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.499987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72d6568b-7488-499f-b4be-e177b12de27c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.501714 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.501694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.501939 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.501919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/72d6568b-7488-499f-b4be-e177b12de27c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.506668 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.506644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krks2\" (UniqueName: \"kubernetes.io/projected/72d6568b-7488-499f-b4be-e177b12de27c-kube-api-access-krks2\") pod \"prometheus-operator-68bcd7f987-kdv8t\" (UID: \"72d6568b-7488-499f-b4be-e177b12de27c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.601695 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.601659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"
Sep 12 00:20:57.918504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:57.918473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t"]
Sep 12 00:20:57.921359 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:20:57.921336    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d6568b_7488_499f_b4be_e177b12de27c.slice/crio-39ad39eb61637549285b5b45e69a590d19e17831cd0b7cedd6f44bb659ee7951 WatchSource:0}: Error finding container 39ad39eb61637549285b5b45e69a590d19e17831cd0b7cedd6f44bb659ee7951: Status 404 returned error can't find the container with id 39ad39eb61637549285b5b45e69a590d19e17831cd0b7cedd6f44bb659ee7951
Sep 12 00:20:58.156801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:58.156693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhbcl" event={"ID":"589333b5-36cc-4fc7-99a3-7ab405c26767","Type":"ContainerStarted","Data":"3fca63aca9ef75c357a90ec95d12553c6f70a75914e34b2a4bd22801c4e6b351"}
Sep 12 00:20:58.156801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:58.156757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhbcl" event={"ID":"589333b5-36cc-4fc7-99a3-7ab405c26767","Type":"ContainerStarted","Data":"404a85b892cccc6dd7fad5e8ae4644d3f8b1173f6452021a9bc6a89a8570d5a9"}
Sep 12 00:20:58.157825 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:58.157800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t" event={"ID":"72d6568b-7488-499f-b4be-e177b12de27c","Type":"ContainerStarted","Data":"39ad39eb61637549285b5b45e69a590d19e17831cd0b7cedd6f44bb659ee7951"}
Sep 12 00:20:58.173240 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:58.173194    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jhbcl" podStartSLOduration=253.20324528 podStartE2EDuration="4m14.173178449s" totalImagesPullingTime="969.933169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:58.172695881 +0000 UTC m=+254.306686107" watchObservedRunningTime="2026-09-12 00:20:58.173178449 +0000 UTC m=+254.307168674"
Sep 12 00:20:59.075044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:20:59.075004    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" podUID="dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 12 00:21:00.167437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:00.167403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t" event={"ID":"72d6568b-7488-499f-b4be-e177b12de27c","Type":"ContainerStarted","Data":"e2b5eb1cda8fb4e38529d8cbbc8bd95c3fc13cbcb5f75ce2cc2347149e7d4190"}
Sep 12 00:21:00.167818 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:00.167454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t" event={"ID":"72d6568b-7488-499f-b4be-e177b12de27c","Type":"ContainerStarted","Data":"20d78fc2ed7c995602cbea25bf104edc1e603d1d52565eb554dc37046b29a1e6"}
Sep 12 00:21:00.182709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:00.182660    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kdv8t" podStartSLOduration=1.746834802 podStartE2EDuration="3.182641991s" totalImagesPullingTime="1.435807189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:00.1822698 +0000 UTC m=+256.316260026" watchObservedRunningTime="2026-09-12 00:21:00.182641991 +0000 UTC m=+256.316632216"
Sep 12 00:21:01.644378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.644341    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qpwgx"]
Sep 12 00:21:01.646978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.646958    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.649321 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.649290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 12 00:21:01.649435 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.649290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j7djq\""
Sep 12 00:21:01.649435 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.649290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 12 00:21:01.650824 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.650804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 12 00:21:01.726610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rx2b\" (UniqueName: \"kubernetes.io/projected/83ad8709-acd6-45d4-953c-1856c229ce21-kube-api-access-2rx2b\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-metrics-client-ca\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-wtmp\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-accelerators-collector-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-textfile\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-root\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.726989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.726850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-sys\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.827955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.827918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.827955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.827958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.827973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rx2b\" (UniqueName: \"kubernetes.io/projected/83ad8709-acd6-45d4-953c-1856c229ce21-kube-api-access-2rx2b\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.827999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-metrics-client-ca\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-wtmp\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-accelerators-collector-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:21:01.828103    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-textfile\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-root\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:21:01.828190    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls podName:83ad8709-acd6-45d4-953c-1856c229ce21 nodeName:}" failed. No retries permitted until 2026-09-12 00:21:02.328167315 +0000 UTC m=+258.462157522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls") pod "node-exporter-qpwgx" (UID: "83ad8709-acd6-45d4-953c-1856c229ce21") : secret "node-exporter-tls" not found
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-wtmp\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-root\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828598 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-sys\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828598 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83ad8709-acd6-45d4-953c-1856c229ce21-sys\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828598 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-textfile\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-metrics-client-ca\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.828734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.828716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-accelerators-collector-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.830451 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.830428    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:01.838807 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:01.838758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rx2b\" (UniqueName: \"kubernetes.io/projected/83ad8709-acd6-45d4-953c-1856c229ce21-kube-api-access-2rx2b\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:02.332361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:02.332326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:02.334636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:02.334606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83ad8709-acd6-45d4-953c-1856c229ce21-node-exporter-tls\") pod \"node-exporter-qpwgx\" (UID: \"83ad8709-acd6-45d4-953c-1856c229ce21\") " pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:02.555321 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:02.555283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qpwgx"
Sep 12 00:21:02.566590 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:21:02.566562    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ad8709_acd6_45d4_953c_1856c229ce21.slice/crio-2a95caee1ca72e692365646f16c4471fb7c9718c860bc81f3f255c017397d812 WatchSource:0}: Error finding container 2a95caee1ca72e692365646f16c4471fb7c9718c860bc81f3f255c017397d812: Status 404 returned error can't find the container with id 2a95caee1ca72e692365646f16c4471fb7c9718c860bc81f3f255c017397d812
Sep 12 00:21:03.179590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.179545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qpwgx" event={"ID":"83ad8709-acd6-45d4-953c-1856c229ce21","Type":"ContainerStarted","Data":"2a95caee1ca72e692365646f16c4471fb7c9718c860bc81f3f255c017397d812"}
Sep 12 00:21:03.626010 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.625936    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"]
Sep 12 00:21:03.629081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.629059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.631857 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.631822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 12 00:21:03.631958 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.631874    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8pp9l6p3lnq1r\""
Sep 12 00:21:03.631958 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.631907    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 12 00:21:03.631958 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.631829    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 12 00:21:03.632115 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.632009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 12 00:21:03.632115 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.632014    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 12 00:21:03.632365 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.632350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6dhjj\""
Sep 12 00:21:03.642682 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.642660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"]
Sep 12 00:21:03.742996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.742964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.742996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.742998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j957\" (UniqueName: \"kubernetes.io/projected/c9b97280-292c-486c-9743-d59408327a15-kube-api-access-7j957\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b97280-292c-486c-9743-d59408327a15-metrics-client-ca\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-grpc-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.743399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.743326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844124 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844124 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j957\" (UniqueName: \"kubernetes.io/projected/c9b97280-292c-486c-9743-d59408327a15-kube-api-access-7j957\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b97280-292c-486c-9743-d59408327a15-metrics-client-ca\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-grpc-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.844600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.844433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.845116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.845065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b97280-292c-486c-9743-d59408327a15-metrics-client-ca\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.846870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.846842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.846984 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.846871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.847194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.847173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.847260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.847178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.847260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.847232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-thanos-querier-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.847372 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.847356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9b97280-292c-486c-9743-d59408327a15-secret-grpc-tls\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.853023 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.853002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j957\" (UniqueName: \"kubernetes.io/projected/c9b97280-292c-486c-9743-d59408327a15-kube-api-access-7j957\") pod \"thanos-querier-5f89f6dc95-57jlh\" (UID: \"c9b97280-292c-486c-9743-d59408327a15\") " pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:03.938506 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:03.938456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:04.184056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:04.184026    2580 generic.go:356] "Generic (PLEG): container finished" podID="83ad8709-acd6-45d4-953c-1856c229ce21" containerID="19da946f856907f3f878feae454eef4360bd85670483db9ec79a570b59c83303" exitCode=0
Sep 12 00:21:04.184427 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:04.184105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qpwgx" event={"ID":"83ad8709-acd6-45d4-953c-1856c229ce21","Type":"ContainerDied","Data":"19da946f856907f3f878feae454eef4360bd85670483db9ec79a570b59c83303"}
Sep 12 00:21:04.210905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:04.210879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"]
Sep 12 00:21:04.218820 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:21:04.218797    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b97280_292c_486c_9743_d59408327a15.slice/crio-2c53ef1614d5e1ee7ba577be6ae7b6db343f5fe49c662db0879d35725e05b85c WatchSource:0}: Error finding container 2c53ef1614d5e1ee7ba577be6ae7b6db343f5fe49c662db0879d35725e05b85c: Status 404 returned error can't find the container with id 2c53ef1614d5e1ee7ba577be6ae7b6db343f5fe49c662db0879d35725e05b85c
Sep 12 00:21:05.189210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.189171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"2c53ef1614d5e1ee7ba577be6ae7b6db343f5fe49c662db0879d35725e05b85c"}
Sep 12 00:21:05.191375 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.191345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qpwgx" event={"ID":"83ad8709-acd6-45d4-953c-1856c229ce21","Type":"ContainerStarted","Data":"ce5509dfdc33a9f66fbb658055b4e2110e141c02f3ee7896ab226c084da6fb12"}
Sep 12 00:21:05.191473 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.191395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qpwgx" event={"ID":"83ad8709-acd6-45d4-953c-1856c229ce21","Type":"ContainerStarted","Data":"36fc94a33c713de47d748d1231dadf7e9007d7978bc15b6605b979449675628e"}
Sep 12 00:21:05.200721 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.200697    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:21:05.203704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.203679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.205985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.205911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 12 00:21:05.206097 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206000    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 12 00:21:05.206097 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206027    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 12 00:21:05.206213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206169    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 00:21:05.206270 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206244    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 12 00:21:05.206270 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206244    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 00:21:05.206578 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bsxwp\""
Sep 12 00:21:05.206659 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.206594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 12 00:21:05.209627 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.209580    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qpwgx" podStartSLOduration=3.085447922 podStartE2EDuration="4.209565189s" totalImagesPullingTime="1.062827694s" totalInitContainerRuntime="61.289573ms" isStatefulPod=true podCreationTimestamp="2026-09-12 00:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:05.208559809 +0000 UTC m=+261.342550036" watchObservedRunningTime="2026-09-12 00:21:05.209565189 +0000 UTC m=+261.343555415"
Sep 12 00:21:05.212666 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.212644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:21:05.358387 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.358575 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.358575 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.358575 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.358730 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.358730 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.358699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gv8\" (UniqueName: \"kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459382 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459382 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459382 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459382 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459701 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.459701 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.459464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gv8\" (UniqueName: \"kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.461421 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.460859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.461421 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.461291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.461594 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.461550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.462526 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.462503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.462640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.462513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.471048 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.471013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gv8\" (UniqueName: \"kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8\") pod \"console-55b4d97747-trx5r\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") " pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.513864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.513835    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:05.786191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:05.786013    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:21:06.178979 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:21:06.178947    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdc4a27_2863_449d_bf16_f02bbd7c164d.slice/crio-39996a7ada1a127f9d28b362f7443288274d49f5d1ee4beae91ca6185463790f WatchSource:0}: Error finding container 39996a7ada1a127f9d28b362f7443288274d49f5d1ee4beae91ca6185463790f: Status 404 returned error can't find the container with id 39996a7ada1a127f9d28b362f7443288274d49f5d1ee4beae91ca6185463790f
Sep 12 00:21:06.197215 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:06.197190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b4d97747-trx5r" event={"ID":"afdc4a27-2863-449d-bf16-f02bbd7c164d","Type":"ContainerStarted","Data":"39996a7ada1a127f9d28b362f7443288274d49f5d1ee4beae91ca6185463790f"}
Sep 12 00:21:07.204693 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:07.204654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"80934c016da184fa3e0cf1ff5edcbace82e7f2f79926d452512aa0cc4fbaa9b2"}
Sep 12 00:21:07.205137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:07.204749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"7abe6d5ea3bec34270e8a994b26d922ff708d217fd540a610c4eb4ec56742086"}
Sep 12 00:21:07.205137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:07.204810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"cd8bdedda80750941febb00d3fca8d80f445068753d51df0db867a0e9f4b14ec"}
Sep 12 00:21:08.215131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:08.213877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"c85da2727fed6f7edf059f027b7d47267848dc4964e5908921f947cf1f7551c2"}
Sep 12 00:21:08.215131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:08.213957    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"0218848b2a96f373f2676b71f6940e3fce5d9e90c8f755eff09e152a9d1e2a12"}
Sep 12 00:21:08.215131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:08.213997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" event={"ID":"c9b97280-292c-486c-9743-d59408327a15","Type":"ContainerStarted","Data":"3543d83a5f071e0273257030582f24cfd8cf7cf54263e5b765127a2c6daba784"}
Sep 12 00:21:08.215131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:08.214044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:08.236372 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:08.236301    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh" podStartSLOduration=2.265419452 podStartE2EDuration="5.236278846s" totalImagesPullingTime="2.970859394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:08.235708072 +0000 UTC m=+264.369698297" watchObservedRunningTime="2026-09-12 00:21:08.236278846 +0000 UTC m=+264.370269071"
Sep 12 00:21:09.075505 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:09.075458    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" podUID="dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 12 00:21:09.075677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:09.075537    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz"
Sep 12 00:21:09.076095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:09.076053    2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6117f69070169e8346c9be56e1f7ebb2b405fe69b278da0a7cf2d4bf03b51222"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 12 00:21:09.076177 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:09.076121    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" podUID="dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29" containerName="service-proxy" containerID="cri-o://6117f69070169e8346c9be56e1f7ebb2b405fe69b278da0a7cf2d4bf03b51222" gracePeriod=30
Sep 12 00:21:10.222929 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:10.222891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b4d97747-trx5r" event={"ID":"afdc4a27-2863-449d-bf16-f02bbd7c164d","Type":"ContainerStarted","Data":"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"}
Sep 12 00:21:10.225137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:10.225114    2580 generic.go:356] "Generic (PLEG): container finished" podID="dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29" containerID="6117f69070169e8346c9be56e1f7ebb2b405fe69b278da0a7cf2d4bf03b51222" exitCode=2
Sep 12 00:21:10.225220 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:10.225164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerDied","Data":"6117f69070169e8346c9be56e1f7ebb2b405fe69b278da0a7cf2d4bf03b51222"}
Sep 12 00:21:10.225220 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:10.225189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bf5f696b6-2pdxz" event={"ID":"dc90d4d5-a0d1-46fe-9f46-e7c5a4f2be29","Type":"ContainerStarted","Data":"0ac62bf2c6b6b17cfa8d7f39c9431c0448270e3c9139628feb56619741d42eea"}
Sep 12 00:21:10.241297 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:10.241251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55b4d97747-trx5r" podStartSLOduration=2.070915887 podStartE2EDuration="5.241234247s" totalImagesPullingTime="3.17031836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:10.240101583 +0000 UTC m=+266.374091808" watchObservedRunningTime="2026-09-12 00:21:10.241234247 +0000 UTC m=+266.375224573"
Sep 12 00:21:14.224740 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:14.224713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f89f6dc95-57jlh"
Sep 12 00:21:15.514482 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:15.514443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:15.514975 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:15.514513    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:21:15.515956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:15.515929    2580 patch_prober.go:28] interesting pod/console-55b4d97747-trx5r container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" start-of-body=
Sep 12 00:21:15.516061 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:15.515975    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-55b4d97747-trx5r" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused"
Sep 12 00:21:23.878695 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:21:23.878651    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xx595" podUID="3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6"
Sep 12 00:21:24.270871 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:24.270839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:25.514609 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:25.514574    2580 patch_prober.go:28] interesting pod/console-55b4d97747-trx5r container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" start-of-body=
Sep 12 00:21:25.515009 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:25.514639    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-55b4d97747-trx5r" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused"
Sep 12 00:21:27.733360 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:27.733317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:27.735543 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:27.735522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6-metrics-tls\") pod \"dns-default-xx595\" (UID: \"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6\") " pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:27.873993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:27.873959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knhwr\""
Sep 12 00:21:27.882610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:27.882592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:28.158905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:28.158830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx595"]
Sep 12 00:21:28.161413 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:21:28.161386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebc64a9_7fc9_40dc_ba57_5c9d62a8a3d6.slice/crio-349fce07ed4b272d544034f4c337c0ecc644b308b844a90671584de79d9e4ea3 WatchSource:0}: Error finding container 349fce07ed4b272d544034f4c337c0ecc644b308b844a90671584de79d9e4ea3: Status 404 returned error can't find the container with id 349fce07ed4b272d544034f4c337c0ecc644b308b844a90671584de79d9e4ea3
Sep 12 00:21:28.285920 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:28.285885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx595" event={"ID":"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6","Type":"ContainerStarted","Data":"349fce07ed4b272d544034f4c337c0ecc644b308b844a90671584de79d9e4ea3"}
Sep 12 00:21:30.294362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:30.294325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx595" event={"ID":"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6","Type":"ContainerStarted","Data":"c0e618c3d5a99f0ae5ce01b27c4f31f581f264ecb6a2fd8ec30d754de1adabb1"}
Sep 12 00:21:30.294779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:30.294375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx595" event={"ID":"3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6","Type":"ContainerStarted","Data":"cefc8e0d4134d76a6efe6a1b843868a651053bd81a76af2e65ab9e1663df54b3"}
Sep 12 00:21:30.294779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:30.294458    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:30.313336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:30.313287    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xx595" podStartSLOduration=251.615518555 podStartE2EDuration="4m13.313268484s" totalImagesPullingTime="1.697749929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:30.312929457 +0000 UTC m=+286.446919685" watchObservedRunningTime="2026-09-12 00:21:30.313268484 +0000 UTC m=+286.447258710"
Sep 12 00:21:34.969211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:34.969178    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:21:40.300187 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:40.300157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xx595"
Sep 12 00:21:44.234642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:44.234617    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 12 00:21:59.988166 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:21:59.988123    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55b4d97747-trx5r" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console" containerID="cri-o://19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831" gracePeriod=15
Sep 12 00:22:00.223614 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.223589    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:22:00.363228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363117    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363179    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42gv8\" (UniqueName: \"kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363497 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363497 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363400    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert\") pod \"afdc4a27-2863-449d-bf16-f02bbd7c164d\" (UID: \"afdc4a27-2863-449d-bf16-f02bbd7c164d\") "
Sep 12 00:22:00.363654 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:00.363880 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config" (OuterVolumeSpecName: "console-config") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:00.363880 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.363663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca" (OuterVolumeSpecName: "service-ca") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:00.365644 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.365611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8" (OuterVolumeSpecName: "kube-api-access-42gv8") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "kube-api-access-42gv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:00.365725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.365655    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:00.365725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.365677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "afdc4a27-2863-449d-bf16-f02bbd7c164d" (UID: "afdc4a27-2863-449d-bf16-f02bbd7c164d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:00.398321 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.398293    2580 generic.go:356] "Generic (PLEG): container finished" podID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerID="19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831" exitCode=0
Sep 12 00:22:00.398425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.398338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b4d97747-trx5r" event={"ID":"afdc4a27-2863-449d-bf16-f02bbd7c164d","Type":"ContainerDied","Data":"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"}
Sep 12 00:22:00.398425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.398352    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b4d97747-trx5r"
Sep 12 00:22:00.398425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.398385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b4d97747-trx5r" event={"ID":"afdc4a27-2863-449d-bf16-f02bbd7c164d","Type":"ContainerDied","Data":"39996a7ada1a127f9d28b362f7443288274d49f5d1ee4beae91ca6185463790f"}
Sep 12 00:22:00.398425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.398412    2580 scope.go:117] "RemoveContainer" containerID="19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"
Sep 12 00:22:00.405969 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.405953    2580 scope.go:117] "RemoveContainer" containerID="19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"
Sep 12 00:22:00.406246 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:22:00.406225    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831\": container with ID starting with 19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831 not found: ID does not exist" containerID="19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"
Sep 12 00:22:00.406287 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.406254    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831"} err="failed to get container status \"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831\": rpc error: code = NotFound desc = could not find container \"19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831\": container with ID starting with 19383ee23228388a12e80afb8e94fd98d6e07f199cca0f15c7365a59851ab831 not found: ID does not exist"
Sep 12 00:22:00.413103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.413081    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:22:00.415405 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.415384    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55b4d97747-trx5r"]
Sep 12 00:22:00.463917 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463886    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-oauth-serving-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.463917 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463913    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42gv8\" (UniqueName: \"kubernetes.io/projected/afdc4a27-2863-449d-bf16-f02bbd7c164d-kube-api-access-42gv8\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.464079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463926    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.464079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463935    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-config\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.464079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463944    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-oauth-config\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.464079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.463952    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdc4a27-2863-449d-bf16-f02bbd7c164d-console-serving-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:22:00.612279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:00.612250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrrp5_7b3ab479-faf4-46d8-af56-b8295bb9c41d/serve-healthcheck-canary/0.log"
Sep 12 00:22:02.372177 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:02.372135    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" path="/var/lib/kubelet/pods/afdc4a27-2863-449d-bf16-f02bbd7c164d/volumes"
Sep 12 00:22:29.953499 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.953454    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:22:29.953995 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.953739    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console"
Sep 12 00:22:29.953995 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.953752    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console"
Sep 12 00:22:29.953995 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.953812    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="afdc4a27-2863-449d-bf16-f02bbd7c164d" containerName="console"
Sep 12 00:22:29.956252 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.956232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:29.958529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.958506    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 00:22:29.958626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.958608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bsxwp\""
Sep 12 00:22:29.960384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.960358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 12 00:22:29.960583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.960560    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 00:22:29.960583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.960579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 12 00:22:29.960935 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.960675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 12 00:22:29.961226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.961206    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 12 00:22:29.961322 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.961227    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 12 00:22:29.965668 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.965651    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 12 00:22:29.968418 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:29.968391    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:22:30.079699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.079665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080028 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.079959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t774m\" (UniqueName: \"kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.080046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080221 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.080157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080221 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.080191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080319 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.080236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.080319 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.080269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t774m\" (UniqueName: \"kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.181871 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.181836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.182057 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.182036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.182163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.182115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.182482 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.182460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.182540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.182480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.184329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.184280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.184656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.184628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.189276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.189252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t774m\" (UniqueName: \"kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m\") pod \"console-56fbbd4698-xsdfd\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") " pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.268515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.268422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:30.543459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:30.543368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:22:30.547097 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:22:30.547055    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d4dba0_6565_4226_a87a_df557c94fbc5.slice/crio-cbcd7f0f73eaa49098902ab294b54bdf48a814597e1d4bfbdcd347aee521ba1e WatchSource:0}: Error finding container cbcd7f0f73eaa49098902ab294b54bdf48a814597e1d4bfbdcd347aee521ba1e: Status 404 returned error can't find the container with id cbcd7f0f73eaa49098902ab294b54bdf48a814597e1d4bfbdcd347aee521ba1e
Sep 12 00:22:31.516037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:31.516001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fbbd4698-xsdfd" event={"ID":"c5d4dba0-6565-4226-a87a-df557c94fbc5","Type":"ContainerStarted","Data":"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"}
Sep 12 00:22:31.516430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:31.516058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fbbd4698-xsdfd" event={"ID":"c5d4dba0-6565-4226-a87a-df557c94fbc5","Type":"ContainerStarted","Data":"cbcd7f0f73eaa49098902ab294b54bdf48a814597e1d4bfbdcd347aee521ba1e"}
Sep 12 00:22:31.536546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:31.536494    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56fbbd4698-xsdfd" podStartSLOduration=2.536476697 podStartE2EDuration="2.536476697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:22:31.535198092 +0000 UTC m=+347.669188317" watchObservedRunningTime="2026-09-12 00:22:31.536476697 +0000 UTC m=+347.670466921"
Sep 12 00:22:40.269164 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:40.269072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:40.269164 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:40.269138    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:40.273779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:40.273742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:22:40.551559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:22:40.551485    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:23:49.216044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:23:49.216011    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:24:14.235330 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.235241    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56fbbd4698-xsdfd" podUID="c5d4dba0-6565-4226-a87a-df557c94fbc5" containerName="console" containerID="cri-o://553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7" gracePeriod=15
Sep 12 00:24:14.469521 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.469496    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:24:14.621827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621726    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.621827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621870    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.621920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t774m\" (UniqueName: \"kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m\") pod \"c5d4dba0-6565-4226-a87a-df557c94fbc5\" (UID: \"c5d4dba0-6565-4226-a87a-df557c94fbc5\") "
Sep 12 00:24:14.622311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.622167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:24:14.622311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.622307    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:24:14.622494 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.622420    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca" (OuterVolumeSpecName: "service-ca") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:24:14.622548 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.622490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config" (OuterVolumeSpecName: "console-config") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:24:14.624064 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.624044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:24:14.624428 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.624407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:24:14.624487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.624453    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m" (OuterVolumeSpecName: "kube-api-access-t774m") pod "c5d4dba0-6565-4226-a87a-df557c94fbc5" (UID: "c5d4dba0-6565-4226-a87a-df557c94fbc5"). InnerVolumeSpecName "kube-api-access-t774m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:24:14.722903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722864    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.722903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722892    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-serving-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.722903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722900    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-oauth-config\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.723119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722909    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t774m\" (UniqueName: \"kubernetes.io/projected/c5d4dba0-6565-4226-a87a-df557c94fbc5-kube-api-access-t774m\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.723119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722921    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-oauth-serving-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.723119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722929    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-console-config\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.723119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.722937    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d4dba0-6565-4226-a87a-df557c94fbc5-trusted-ca-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:24:14.877882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.877801    2580 generic.go:356] "Generic (PLEG): container finished" podID="c5d4dba0-6565-4226-a87a-df557c94fbc5" containerID="553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7" exitCode=0
Sep 12 00:24:14.877882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.877846    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fbbd4698-xsdfd"
Sep 12 00:24:14.877882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.877877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fbbd4698-xsdfd" event={"ID":"c5d4dba0-6565-4226-a87a-df557c94fbc5","Type":"ContainerDied","Data":"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"}
Sep 12 00:24:14.878074 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.877914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fbbd4698-xsdfd" event={"ID":"c5d4dba0-6565-4226-a87a-df557c94fbc5","Type":"ContainerDied","Data":"cbcd7f0f73eaa49098902ab294b54bdf48a814597e1d4bfbdcd347aee521ba1e"}
Sep 12 00:24:14.878074 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.877932    2580 scope.go:117] "RemoveContainer" containerID="553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"
Sep 12 00:24:14.885829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.885801    2580 scope.go:117] "RemoveContainer" containerID="553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"
Sep 12 00:24:14.886163 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:24:14.886139    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7\": container with ID starting with 553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7 not found: ID does not exist" containerID="553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"
Sep 12 00:24:14.886235 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.886172    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7"} err="failed to get container status \"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7\": rpc error: code = NotFound desc = could not find container \"553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7\": container with ID starting with 553fb1da39cafb9433790e3b66ff8e4ee4b8cf725256382c8365914bfb4537c7 not found: ID does not exist"
Sep 12 00:24:14.897087 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.897060    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:24:14.900980 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:14.900960    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56fbbd4698-xsdfd"]
Sep 12 00:24:16.372579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:16.372543    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d4dba0-6565-4226-a87a-df557c94fbc5" path="/var/lib/kubelet/pods/c5d4dba0-6565-4226-a87a-df557c94fbc5/volumes"
Sep 12 00:24:51.133639 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.133598    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"]
Sep 12 00:24:51.134182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.133975    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d4dba0-6565-4226-a87a-df557c94fbc5" containerName="console"
Sep 12 00:24:51.134182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.133991    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d4dba0-6565-4226-a87a-df557c94fbc5" containerName="console"
Sep 12 00:24:51.134182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.134068    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d4dba0-6565-4226-a87a-df557c94fbc5" containerName="console"
Sep 12 00:24:51.138193 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.138171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.140540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.140516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 00:24:51.140644 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.140531    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 00:24:51.141244 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.141228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ft8qr\""
Sep 12 00:24:51.144917 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.144613    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"]
Sep 12 00:24:51.182380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.182350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.182555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.182386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.182555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.182421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85ntm\" (UniqueName: \"kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.283331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.283282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.283509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.283346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85ntm\" (UniqueName: \"kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.283509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.283388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.283697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.283675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.283737 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.283698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.291079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.291055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85ntm\" (UniqueName: \"kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.447407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.447376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:24:51.714237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.714158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"]
Sep 12 00:24:51.717390 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:24:51.717364    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5603e61_eabd_459d_a251_1519174447ac.slice/crio-c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf WatchSource:0}: Error finding container c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf: Status 404 returned error can't find the container with id c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf
Sep 12 00:24:51.719365 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:51.719349    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:24:52.006021 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:52.005928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh" event={"ID":"d5603e61-eabd-459d-a251-1519174447ac","Type":"ContainerStarted","Data":"c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf"}
Sep 12 00:24:59.032128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:59.032092    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5603e61-eabd-459d-a251-1519174447ac" containerID="d4a34392890c5cf6727de0afec979884ecafc3cf1b6526b8975150c8fee1c3ca" exitCode=0
Sep 12 00:24:59.032523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:24:59.032172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh" event={"ID":"d5603e61-eabd-459d-a251-1519174447ac","Type":"ContainerDied","Data":"d4a34392890c5cf6727de0afec979884ecafc3cf1b6526b8975150c8fee1c3ca"}
Sep 12 00:25:01.040850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:01.040817    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5603e61-eabd-459d-a251-1519174447ac" containerID="0834011f8825411318e5bcec0ccf8d9ff931ebd2281db7b83a5ad19ec1c86b48" exitCode=0
Sep 12 00:25:01.041235 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:01.040889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh" event={"ID":"d5603e61-eabd-459d-a251-1519174447ac","Type":"ContainerDied","Data":"0834011f8825411318e5bcec0ccf8d9ff931ebd2281db7b83a5ad19ec1c86b48"}
Sep 12 00:25:09.075394 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:09.075359    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5603e61-eabd-459d-a251-1519174447ac" containerID="643e003b42dcb3889c3cd5df9ed4361beda978a883ac32896269b447e4036200" exitCode=0
Sep 12 00:25:09.075394 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:09.075397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh" event={"ID":"d5603e61-eabd-459d-a251-1519174447ac","Type":"ContainerDied","Data":"643e003b42dcb3889c3cd5df9ed4361beda978a883ac32896269b447e4036200"}
Sep 12 00:25:10.201042 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.201019    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:25:10.340032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.339926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle\") pod \"d5603e61-eabd-459d-a251-1519174447ac\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") "
Sep 12 00:25:10.340032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.339983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util\") pod \"d5603e61-eabd-459d-a251-1519174447ac\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") "
Sep 12 00:25:10.340249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.340084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85ntm\" (UniqueName: \"kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm\") pod \"d5603e61-eabd-459d-a251-1519174447ac\" (UID: \"d5603e61-eabd-459d-a251-1519174447ac\") "
Sep 12 00:25:10.340566 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.340542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle" (OuterVolumeSpecName: "bundle") pod "d5603e61-eabd-459d-a251-1519174447ac" (UID: "d5603e61-eabd-459d-a251-1519174447ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:10.342299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.342269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm" (OuterVolumeSpecName: "kube-api-access-85ntm") pod "d5603e61-eabd-459d-a251-1519174447ac" (UID: "d5603e61-eabd-459d-a251-1519174447ac"). InnerVolumeSpecName "kube-api-access-85ntm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:25:10.344555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.344529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util" (OuterVolumeSpecName: "util") pod "d5603e61-eabd-459d-a251-1519174447ac" (UID: "d5603e61-eabd-459d-a251-1519174447ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:10.441268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.441242    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85ntm\" (UniqueName: \"kubernetes.io/projected/d5603e61-eabd-459d-a251-1519174447ac-kube-api-access-85ntm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:10.441268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.441265    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:10.441407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:10.441274    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5603e61-eabd-459d-a251-1519174447ac-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:11.085216 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:11.085178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh" event={"ID":"d5603e61-eabd-459d-a251-1519174447ac","Type":"ContainerDied","Data":"c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf"}
Sep 12 00:25:11.085216 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:11.085211    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wrvjh"
Sep 12 00:25:11.085216 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:11.085226    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9150980c0a92f86daa517506365be15a498e4e6332a504518cee4fabf636baf"
Sep 12 00:25:14.023144 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023110    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"]
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023380    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="extract"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023392    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="extract"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023406    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="util"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023412    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="util"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023419    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="pull"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023424    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="pull"
Sep 12 00:25:14.023587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.023464    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5603e61-eabd-459d-a251-1519174447ac" containerName="extract"
Sep 12 00:25:14.057374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.057350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"]
Sep 12 00:25:14.057525 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.057444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.060513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.060490    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qtwbd\""
Sep 12 00:25:14.061347 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.061326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:25:14.061450 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.061368    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 12 00:25:14.168614 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.168573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlmsb\" (UniqueName: \"kubernetes.io/projected/423d1366-e338-4594-bf5a-627e21be3497-kube-api-access-xlmsb\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.168819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.168683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/423d1366-e338-4594-bf5a-627e21be3497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.269446 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.269413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/423d1366-e338-4594-bf5a-627e21be3497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.269642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.269454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlmsb\" (UniqueName: \"kubernetes.io/projected/423d1366-e338-4594-bf5a-627e21be3497-kube-api-access-xlmsb\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.269815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.269795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/423d1366-e338-4594-bf5a-627e21be3497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.277122 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.277066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlmsb\" (UniqueName: \"kubernetes.io/projected/423d1366-e338-4594-bf5a-627e21be3497-kube-api-access-xlmsb\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqb88\" (UID: \"423d1366-e338-4594-bf5a-627e21be3497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.366452 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.366422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"
Sep 12 00:25:14.666054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:14.665965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88"]
Sep 12 00:25:14.669874 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:14.669846    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423d1366_e338_4594_bf5a_627e21be3497.slice/crio-27346b97ec587c465a37f0bdd02c843530108eea8988cba7f30e2559f4efb70d WatchSource:0}: Error finding container 27346b97ec587c465a37f0bdd02c843530108eea8988cba7f30e2559f4efb70d: Status 404 returned error can't find the container with id 27346b97ec587c465a37f0bdd02c843530108eea8988cba7f30e2559f4efb70d
Sep 12 00:25:15.102533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:15.102498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88" event={"ID":"423d1366-e338-4594-bf5a-627e21be3497","Type":"ContainerStarted","Data":"27346b97ec587c465a37f0bdd02c843530108eea8988cba7f30e2559f4efb70d"}
Sep 12 00:25:17.112159 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.112118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88" event={"ID":"423d1366-e338-4594-bf5a-627e21be3497","Type":"ContainerStarted","Data":"e344c2cedf644c64eac4406e47e635b7d6d4b242660eeabdf5ab60bc1aedb873"}
Sep 12 00:25:17.135856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.135809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqb88" podStartSLOduration=1.688564008 podStartE2EDuration="3.135792393s" totalImagesPullingTime="1.447228385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:25:17.133949 +0000 UTC m=+513.267939226" watchObservedRunningTime="2026-09-12 00:25:17.135792393 +0000 UTC m=+513.269782618"
Sep 12 00:25:17.985445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.985410    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"]
Sep 12 00:25:17.987904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.987881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:17.990667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.990648    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 00:25:17.991487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.991467    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ft8qr\""
Sep 12 00:25:17.991722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:17.991705    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 00:25:18.001212 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.001190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"]
Sep 12 00:25:18.100237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.100195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.100413 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.100241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbxpf\" (UniqueName: \"kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.100413 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.100327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.200943 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.200908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.201407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.200955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.201407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.200978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbxpf\" (UniqueName: \"kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.201407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.201299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.201548 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.201406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.209736 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.209705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbxpf\" (UniqueName: \"kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.297344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.297272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:18.566785 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:18.566686    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"]
Sep 12 00:25:18.569603 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:18.569575    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1069a3_c038_431d_938e_a5a01042057f.slice/crio-9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914 WatchSource:0}: Error finding container 9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914: Status 404 returned error can't find the container with id 9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914
Sep 12 00:25:19.122558 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:19.122518    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d1069a3-c038-431d-938e-a5a01042057f" containerID="61bc7ae6f14678318f8b126a7527d578e462fe6e7a690ae57c1ee5441b733121" exitCode=0
Sep 12 00:25:19.122795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:19.122604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg" event={"ID":"3d1069a3-c038-431d-938e-a5a01042057f","Type":"ContainerDied","Data":"61bc7ae6f14678318f8b126a7527d578e462fe6e7a690ae57c1ee5441b733121"}
Sep 12 00:25:19.122795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:19.122656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg" event={"ID":"3d1069a3-c038-431d-938e-a5a01042057f","Type":"ContainerStarted","Data":"9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914"}
Sep 12 00:25:22.136706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:22.136668    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d1069a3-c038-431d-938e-a5a01042057f" containerID="e26ed3dc4286b4fa4165b67187e34ff121f05949fddd77d6da7deff8b2eff352" exitCode=0
Sep 12 00:25:22.137107 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:22.136760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg" event={"ID":"3d1069a3-c038-431d-938e-a5a01042057f","Type":"ContainerDied","Data":"e26ed3dc4286b4fa4165b67187e34ff121f05949fddd77d6da7deff8b2eff352"}
Sep 12 00:25:23.142038 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:23.142005    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d1069a3-c038-431d-938e-a5a01042057f" containerID="731714bdac9e69116f16e01145e88f569a705fcc7106dd6623666a57318f495c" exitCode=0
Sep 12 00:25:23.142410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:23.142099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg" event={"ID":"3d1069a3-c038-431d-938e-a5a01042057f","Type":"ContainerDied","Data":"731714bdac9e69116f16e01145e88f569a705fcc7106dd6623666a57318f495c"}
Sep 12 00:25:24.264956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.264931    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:24.349304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.349276    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbxpf\" (UniqueName: \"kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf\") pod \"3d1069a3-c038-431d-938e-a5a01042057f\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") "
Sep 12 00:25:24.349437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.349353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle\") pod \"3d1069a3-c038-431d-938e-a5a01042057f\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") "
Sep 12 00:25:24.349437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.349393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util\") pod \"3d1069a3-c038-431d-938e-a5a01042057f\" (UID: \"3d1069a3-c038-431d-938e-a5a01042057f\") "
Sep 12 00:25:24.349895 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.349859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle" (OuterVolumeSpecName: "bundle") pod "3d1069a3-c038-431d-938e-a5a01042057f" (UID: "3d1069a3-c038-431d-938e-a5a01042057f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:24.351326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.351296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf" (OuterVolumeSpecName: "kube-api-access-gbxpf") pod "3d1069a3-c038-431d-938e-a5a01042057f" (UID: "3d1069a3-c038-431d-938e-a5a01042057f"). InnerVolumeSpecName "kube-api-access-gbxpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:25:24.354328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.354291    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util" (OuterVolumeSpecName: "util") pod "3d1069a3-c038-431d-938e-a5a01042057f" (UID: "3d1069a3-c038-431d-938e-a5a01042057f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:24.450158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.450125    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:24.450158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.450156    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbxpf\" (UniqueName: \"kubernetes.io/projected/3d1069a3-c038-431d-938e-a5a01042057f-kube-api-access-gbxpf\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:24.450357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:24.450170    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d1069a3-c038-431d-938e-a5a01042057f-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:25.152345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.152306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg" event={"ID":"3d1069a3-c038-431d-938e-a5a01042057f","Type":"ContainerDied","Data":"9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914"}
Sep 12 00:25:25.152345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.152352    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f581f95cd6d525988ad846d491efe8f6f9ebafc4d1cf9367e6d10087e15b914"
Sep 12 00:25:25.152626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.152315    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fph9fg"
Sep 12 00:25:25.669684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669648    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qsmx"]
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669914    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="pull"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669924    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="pull"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669934    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="extract"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669941    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="extract"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669949    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="util"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.669955    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="util"
Sep 12 00:25:25.670069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.670004    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d1069a3-c038-431d-938e-a5a01042057f" containerName="extract"
Sep 12 00:25:25.671679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.671661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.673832 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.673812    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 12 00:25:25.674492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.674473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 12 00:25:25.674492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.674485    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-x84xw\""
Sep 12 00:25:25.679954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.679934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qsmx"]
Sep 12 00:25:25.759585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.759549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.759783 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.759606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vx7p\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-kube-api-access-4vx7p\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.860370 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.860339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vx7p\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-kube-api-access-4vx7p\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.860518 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.860391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.867726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.867697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.868174 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.868159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vx7p\" (UniqueName: \"kubernetes.io/projected/5bbd65da-979f-4e22-ba43-e6c415a90fba-kube-api-access-4vx7p\") pod \"cert-manager-webhook-6486ffd997-8qsmx\" (UID: \"5bbd65da-979f-4e22-ba43-e6c415a90fba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:25.980374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:25.980287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:26.246982 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:26.246911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qsmx"]
Sep 12 00:25:26.249812 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:26.249781    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bbd65da_979f_4e22_ba43_e6c415a90fba.slice/crio-945a4be26e145d2ad3b1ba3bdcd60e56e699c58067fd9adffaf62c97fc8460c1 WatchSource:0}: Error finding container 945a4be26e145d2ad3b1ba3bdcd60e56e699c58067fd9adffaf62c97fc8460c1: Status 404 returned error can't find the container with id 945a4be26e145d2ad3b1ba3bdcd60e56e699c58067fd9adffaf62c97fc8460c1
Sep 12 00:25:27.162452 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:27.162415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx" event={"ID":"5bbd65da-979f-4e22-ba43-e6c415a90fba","Type":"ContainerStarted","Data":"945a4be26e145d2ad3b1ba3bdcd60e56e699c58067fd9adffaf62c97fc8460c1"}
Sep 12 00:25:28.755230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.755203    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hjzp"]
Sep 12 00:25:28.757646 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.757628    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.759669 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.759651    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zs2np\""
Sep 12 00:25:28.767644 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.767618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hjzp"]
Sep 12 00:25:28.883420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.883382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjlnd\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-kube-api-access-fjlnd\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.883589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.883431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.984194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.984103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjlnd\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-kube-api-access-fjlnd\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.984194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.984145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.991836 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.991812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:28.992056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:28.992033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjlnd\" (UniqueName: \"kubernetes.io/projected/b36ccfef-51f1-4ae8-95e9-6b71e316c0b1-kube-api-access-fjlnd\") pod \"cert-manager-55b4d954d5-2hjzp\" (UID: \"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1\") " pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:29.082351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:29.082322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2hjzp"
Sep 12 00:25:29.172054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:29.172015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx" event={"ID":"5bbd65da-979f-4e22-ba43-e6c415a90fba","Type":"ContainerStarted","Data":"4b31add60975e366d4f8390d1ae2cb2ccb69cfecd13d20d7ef84bdd5bef34456"}
Sep 12 00:25:29.172199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:29.172082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:29.187581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:29.187536    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx" podStartSLOduration=1.7064287070000002 podStartE2EDuration="4.187521609s" totalImagesPullingTime="2.481092902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:25:29.187032267 +0000 UTC m=+525.321022493" watchObservedRunningTime="2026-09-12 00:25:29.187521609 +0000 UTC m=+525.321511834"
Sep 12 00:25:29.352838 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:29.352740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hjzp"]
Sep 12 00:25:29.355584 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:29.355558    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb36ccfef_51f1_4ae8_95e9_6b71e316c0b1.slice/crio-bf3c0d7300b2415c203de4d2503966f07c04dc484b90b17157021db3e2570426 WatchSource:0}: Error finding container bf3c0d7300b2415c203de4d2503966f07c04dc484b90b17157021db3e2570426: Status 404 returned error can't find the container with id bf3c0d7300b2415c203de4d2503966f07c04dc484b90b17157021db3e2570426
Sep 12 00:25:30.177209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:30.177175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2hjzp" event={"ID":"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1","Type":"ContainerStarted","Data":"3783c4000f140d1aa58db02feec6701234a0ecbb3f13d81ec987ee46c9cfe182"}
Sep 12 00:25:30.177209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:30.177219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2hjzp" event={"ID":"b36ccfef-51f1-4ae8-95e9-6b71e316c0b1","Type":"ContainerStarted","Data":"bf3c0d7300b2415c203de4d2503966f07c04dc484b90b17157021db3e2570426"}
Sep 12 00:25:30.200887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:30.200840    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2hjzp" podStartSLOduration=2.200825223 podStartE2EDuration="2.200825223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:25:30.199211641 +0000 UTC m=+526.333201875" watchObservedRunningTime="2026-09-12 00:25:30.200825223 +0000 UTC m=+526.334815448"
Sep 12 00:25:35.180147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:35.180112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8qsmx"
Sep 12 00:25:37.947533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.947451    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"]
Sep 12 00:25:37.949580 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.949562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:37.951775 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.951732    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 00:25:37.952552 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.952532    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ft8qr\""
Sep 12 00:25:37.952641 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.952562    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 00:25:37.958528 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:37.958503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"]
Sep 12 00:25:38.046674 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.046641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.046861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.046683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.046861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.046715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j8lb\" (UniqueName: \"kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.148039 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.147994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.148210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.148047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.148210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.148117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j8lb\" (UniqueName: \"kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.148474 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.148454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.148537 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.148480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.155680 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.155642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j8lb\" (UniqueName: \"kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.259544 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.259454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:38.528401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:38.528329    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"]
Sep 12 00:25:38.531697 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:38.531675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20734877_5684_439d_ae83_caec2f0be85f.slice/crio-5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad WatchSource:0}: Error finding container 5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad: Status 404 returned error can't find the container with id 5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad
Sep 12 00:25:39.213735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:39.213698    2580 generic.go:356] "Generic (PLEG): container finished" podID="20734877-5684-439d-ae83-caec2f0be85f" containerID="5b53bee8b8658e018e40ad6b777618db1eb98a0e4d34425719024978f5df89c2" exitCode=0
Sep 12 00:25:39.214128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:39.213800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh" event={"ID":"20734877-5684-439d-ae83-caec2f0be85f","Type":"ContainerDied","Data":"5b53bee8b8658e018e40ad6b777618db1eb98a0e4d34425719024978f5df89c2"}
Sep 12 00:25:39.214128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:39.213852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh" event={"ID":"20734877-5684-439d-ae83-caec2f0be85f","Type":"ContainerStarted","Data":"5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad"}
Sep 12 00:25:40.220094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:40.220061    2580 generic.go:356] "Generic (PLEG): container finished" podID="20734877-5684-439d-ae83-caec2f0be85f" containerID="3552e8e2abc7c2cf9b19c70119c4edae4be1b7183e4cf0a5ee36627890ad1a74" exitCode=0
Sep 12 00:25:40.220499 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:40.220123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh" event={"ID":"20734877-5684-439d-ae83-caec2f0be85f","Type":"ContainerDied","Data":"3552e8e2abc7c2cf9b19c70119c4edae4be1b7183e4cf0a5ee36627890ad1a74"}
Sep 12 00:25:41.226237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:41.226197    2580 generic.go:356] "Generic (PLEG): container finished" podID="20734877-5684-439d-ae83-caec2f0be85f" containerID="e385f09a155c284191cd228c59342d7a1658517458021588fee12afe44ed4f62" exitCode=0
Sep 12 00:25:41.226618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:41.226289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh" event={"ID":"20734877-5684-439d-ae83-caec2f0be85f","Type":"ContainerDied","Data":"e385f09a155c284191cd228c59342d7a1658517458021588fee12afe44ed4f62"}
Sep 12 00:25:42.375369 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.375340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:42.484281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.484196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle\") pod \"20734877-5684-439d-ae83-caec2f0be85f\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") "
Sep 12 00:25:42.484281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.484252    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util\") pod \"20734877-5684-439d-ae83-caec2f0be85f\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") "
Sep 12 00:25:42.484465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.484286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j8lb\" (UniqueName: \"kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb\") pod \"20734877-5684-439d-ae83-caec2f0be85f\" (UID: \"20734877-5684-439d-ae83-caec2f0be85f\") "
Sep 12 00:25:42.485098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.485068    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle" (OuterVolumeSpecName: "bundle") pod "20734877-5684-439d-ae83-caec2f0be85f" (UID: "20734877-5684-439d-ae83-caec2f0be85f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:42.486369 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.486348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb" (OuterVolumeSpecName: "kube-api-access-2j8lb") pod "20734877-5684-439d-ae83-caec2f0be85f" (UID: "20734877-5684-439d-ae83-caec2f0be85f"). InnerVolumeSpecName "kube-api-access-2j8lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:25:42.489992 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.489974    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util" (OuterVolumeSpecName: "util") pod "20734877-5684-439d-ae83-caec2f0be85f" (UID: "20734877-5684-439d-ae83-caec2f0be85f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:42.585605 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.585566    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:42.585605 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.585598    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j8lb\" (UniqueName: \"kubernetes.io/projected/20734877-5684-439d-ae83-caec2f0be85f-kube-api-access-2j8lb\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:42.585605 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:42.585610    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20734877-5684-439d-ae83-caec2f0be85f-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:43.236560 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:43.236533    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh"
Sep 12 00:25:43.236727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:43.236559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxjvlh" event={"ID":"20734877-5684-439d-ae83-caec2f0be85f","Type":"ContainerDied","Data":"5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad"}
Sep 12 00:25:43.236727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:43.236601    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a7c56f4dff427f9fa29895364276c0e0d6782087b03d2f3975439dca5382fad"
Sep 12 00:25:54.133048 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133009    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"]
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133393    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="util"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133413    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="util"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133426    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="extract"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133434    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="extract"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133445    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="pull"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="pull"
Sep 12 00:25:54.133653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.133539    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20734877-5684-439d-ae83-caec2f0be85f" containerName="extract"
Sep 12 00:25:54.138044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.138021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.140503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.140482    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 00:25:54.140503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.140507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 00:25:54.140660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.140633    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ft8qr\""
Sep 12 00:25:54.144127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.144096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"]
Sep 12 00:25:54.274745 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.274707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.274938 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.274784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czw7c\" (UniqueName: \"kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.274938 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.274818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.376108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.376080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czw7c\" (UniqueName: \"kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.376283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.376120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.376283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.376156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.376460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.376443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.376550 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.376500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.394524 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.394462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czw7c\" (UniqueName: \"kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.447595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.447561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:54.718252 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:54.718227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"]
Sep 12 00:25:54.720426 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:54.720401    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc11e2f_2365_47cc_b258_a56d4980536e.slice/crio-0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a WatchSource:0}: Error finding container 0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a: Status 404 returned error can't find the container with id 0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a
Sep 12 00:25:55.287489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.287454    2580 generic.go:356] "Generic (PLEG): container finished" podID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerID="1d7aa766c87c2dbad4f16e2d466b021cd37108ab7d62f046e6a6a0e9f0a9774b" exitCode=0
Sep 12 00:25:55.287887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.287506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c" event={"ID":"6dc11e2f-2365-47cc-b258-a56d4980536e","Type":"ContainerDied","Data":"1d7aa766c87c2dbad4f16e2d466b021cd37108ab7d62f046e6a6a0e9f0a9774b"}
Sep 12 00:25:55.287887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.287556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c" event={"ID":"6dc11e2f-2365-47cc-b258-a56d4980536e","Type":"ContainerStarted","Data":"0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a"}
Sep 12 00:25:55.524629 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.524599    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"]
Sep 12 00:25:55.530544 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.530520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.532693 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.532668    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 12 00:25:55.538650 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.533530    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 12 00:25:55.538650 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.533801    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 12 00:25:55.538650 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.534028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 12 00:25:55.538650 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.534091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mb7xc\""
Sep 12 00:25:55.543815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.543791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"]
Sep 12 00:25:55.585405 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.585375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.585564 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.585415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.585564 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.585441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.585564 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.585472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.585564 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.585510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qdq7\" (UniqueName: \"kubernetes.io/projected/2dd846bb-a92d-4c58-bde8-a96595141003-kube-api-access-9qdq7\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.686559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.686559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.686752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.686752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.686874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qdq7\" (UniqueName: \"kubernetes.io/projected/2dd846bb-a92d-4c58-bde8-a96595141003-kube-api-access-9qdq7\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.687002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.686980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.688914 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.688891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dd846bb-a92d-4c58-bde8-a96595141003-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.689071 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.689038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.689225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.689207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dd846bb-a92d-4c58-bde8-a96595141003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.696291 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.696269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qdq7\" (UniqueName: \"kubernetes.io/projected/2dd846bb-a92d-4c58-bde8-a96595141003-kube-api-access-9qdq7\") pod \"opendatahub-operator-controller-manager-8446f459d6-qpsx4\" (UID: \"2dd846bb-a92d-4c58-bde8-a96595141003\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:55.846589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:55.846513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:25:56.137460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:56.137435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"]
Sep 12 00:25:56.140157 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:25:56.140127    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd846bb_a92d_4c58_bde8_a96595141003.slice/crio-0fa0424e1f4ed16b1718d7d04ef0d276d5d73eb24a3839b926d6851c4706f9d7 WatchSource:0}: Error finding container 0fa0424e1f4ed16b1718d7d04ef0d276d5d73eb24a3839b926d6851c4706f9d7: Status 404 returned error can't find the container with id 0fa0424e1f4ed16b1718d7d04ef0d276d5d73eb24a3839b926d6851c4706f9d7
Sep 12 00:25:56.293407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:56.293362    2580 generic.go:356] "Generic (PLEG): container finished" podID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerID="df01ccc152786dab4e9b3a330337c9cb1678687641530df101ad10e433c6d4df" exitCode=0
Sep 12 00:25:56.293833 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:56.293415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c" event={"ID":"6dc11e2f-2365-47cc-b258-a56d4980536e","Type":"ContainerDied","Data":"df01ccc152786dab4e9b3a330337c9cb1678687641530df101ad10e433c6d4df"}
Sep 12 00:25:56.294974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:56.294890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4" event={"ID":"2dd846bb-a92d-4c58-bde8-a96595141003","Type":"ContainerStarted","Data":"0fa0424e1f4ed16b1718d7d04ef0d276d5d73eb24a3839b926d6851c4706f9d7"}
Sep 12 00:25:57.305023 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:57.304916    2580 generic.go:356] "Generic (PLEG): container finished" podID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerID="355dcad687ccd3c3685da96d9ba42cb29b6a59efeff4abf8998119428a50d80a" exitCode=0
Sep 12 00:25:57.305481 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:57.305037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c" event={"ID":"6dc11e2f-2365-47cc-b258-a56d4980536e","Type":"ContainerDied","Data":"355dcad687ccd3c3685da96d9ba42cb29b6a59efeff4abf8998119428a50d80a"}
Sep 12 00:25:59.241685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.241660    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:59.314809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.314788    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c"
Sep 12 00:25:59.314809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.314788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sg66c" event={"ID":"6dc11e2f-2365-47cc-b258-a56d4980536e","Type":"ContainerDied","Data":"0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a"}
Sep 12 00:25:59.314979 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.314846    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c39fc3c799fbc406c1b24d74f8bb8449ffa65aed9421fd7cd5942dcf8dfe78a"
Sep 12 00:25:59.316908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.316889    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czw7c\" (UniqueName: \"kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c\") pod \"6dc11e2f-2365-47cc-b258-a56d4980536e\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") "
Sep 12 00:25:59.316999 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.316933    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util\") pod \"6dc11e2f-2365-47cc-b258-a56d4980536e\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") "
Sep 12 00:25:59.316999 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.316975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle\") pod \"6dc11e2f-2365-47cc-b258-a56d4980536e\" (UID: \"6dc11e2f-2365-47cc-b258-a56d4980536e\") "
Sep 12 00:25:59.317968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.317942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle" (OuterVolumeSpecName: "bundle") pod "6dc11e2f-2365-47cc-b258-a56d4980536e" (UID: "6dc11e2f-2365-47cc-b258-a56d4980536e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:59.318920 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.318896    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c" (OuterVolumeSpecName: "kube-api-access-czw7c") pod "6dc11e2f-2365-47cc-b258-a56d4980536e" (UID: "6dc11e2f-2365-47cc-b258-a56d4980536e"). InnerVolumeSpecName "kube-api-access-czw7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:25:59.324680 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.324658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util" (OuterVolumeSpecName: "util") pod "6dc11e2f-2365-47cc-b258-a56d4980536e" (UID: "6dc11e2f-2365-47cc-b258-a56d4980536e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:25:59.417927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.417893    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czw7c\" (UniqueName: \"kubernetes.io/projected/6dc11e2f-2365-47cc-b258-a56d4980536e-kube-api-access-czw7c\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:59.417927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.417918    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:25:59.417927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:25:59.417928    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6dc11e2f-2365-47cc-b258-a56d4980536e-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:26:00.322126 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:00.322078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4" event={"ID":"2dd846bb-a92d-4c58-bde8-a96595141003","Type":"ContainerStarted","Data":"3bdbc7f8688f6bfc652340e5eccf6ae1a6863a40901dbec9995aa863bed317cb"}
Sep 12 00:26:01.327946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:01.327910    2580 generic.go:356] "Generic (PLEG): container finished" podID="2dd846bb-a92d-4c58-bde8-a96595141003" containerID="3bdbc7f8688f6bfc652340e5eccf6ae1a6863a40901dbec9995aa863bed317cb" exitCode=0
Sep 12 00:26:01.328323 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:01.327994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4" event={"ID":"2dd846bb-a92d-4c58-bde8-a96595141003","Type":"ContainerDied","Data":"3bdbc7f8688f6bfc652340e5eccf6ae1a6863a40901dbec9995aa863bed317cb"}
Sep 12 00:26:02.334221 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:02.334182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4" event={"ID":"2dd846bb-a92d-4c58-bde8-a96595141003","Type":"ContainerStarted","Data":"65799071c38e4840f8ec0d6467d3b5f1be9f86858fe6d353767537a1ba027020"}
Sep 12 00:26:02.334642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:02.334278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:26:02.355891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:02.355838    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4" podStartSLOduration=2.753278115 podStartE2EDuration="7.355819047s" totalImagesPullingTime="3.536373493s" totalInitContainerRuntime="1.066167439s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:25:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:02.353736462 +0000 UTC m=+558.487726689" watchObservedRunningTime="2026-09-12 00:26:02.355819047 +0000 UTC m=+558.489809272"
Sep 12 00:26:09.045785 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.045731    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f7c784bcf-w5sjw"]
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046165    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="extract"
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046184    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="extract"
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046205    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="pull"
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046213    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="pull"
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046227    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="util"
Sep 12 00:26:09.046258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046234    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="util"
Sep 12 00:26:09.046557 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.046299    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dc11e2f-2365-47cc-b258-a56d4980536e" containerName="extract"
Sep 12 00:26:09.048910 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.048889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.051059 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.051035    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bsxwp\""
Sep 12 00:26:09.052128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052102    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 12 00:26:09.052128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052118    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 12 00:26:09.052267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 12 00:26:09.052267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052204    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 00:26:09.052267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 12 00:26:09.052267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052102    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 00:26:09.052419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.052105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 12 00:26:09.056171 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.056153    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 12 00:26:09.057963 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.057941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f7c784bcf-w5sjw"]
Sep 12 00:26:09.088743 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-oauth-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.088912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7dps\" (UniqueName: \"kubernetes.io/projected/13f345ef-1e12-402d-9577-38b71e30e676-kube-api-access-j7dps\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.088912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-oauth-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.088912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-console-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.088912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-trusted-ca-bundle\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.088912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.089106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.088996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-service-ca\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189327 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-service-ca\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-oauth-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7dps\" (UniqueName: \"kubernetes.io/projected/13f345ef-1e12-402d-9577-38b71e30e676-kube-api-access-j7dps\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-oauth-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-console-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-trusted-ca-bundle\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.189539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.189525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.190237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.190199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-service-ca\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.190237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.190199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-oauth-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.190410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.190202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-console-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.190697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.190672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f345ef-1e12-402d-9577-38b71e30e676-trusted-ca-bundle\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.192064 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.192037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-serving-cert\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.192064 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.192060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f345ef-1e12-402d-9577-38b71e30e676-console-oauth-config\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.196588 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.196554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7dps\" (UniqueName: \"kubernetes.io/projected/13f345ef-1e12-402d-9577-38b71e30e676-kube-api-access-j7dps\") pod \"console-6f7c784bcf-w5sjw\" (UID: \"13f345ef-1e12-402d-9577-38b71e30e676\") " pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.358649 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.358552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:09.646826 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:09.646800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f7c784bcf-w5sjw"]
Sep 12 00:26:09.648819 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:26:09.648786    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f345ef_1e12_402d_9577_38b71e30e676.slice/crio-973d31fb79837b59c29375db525db6b898d9e647b3781cfada6e980e5be1d69f WatchSource:0}: Error finding container 973d31fb79837b59c29375db525db6b898d9e647b3781cfada6e980e5be1d69f: Status 404 returned error can't find the container with id 973d31fb79837b59c29375db525db6b898d9e647b3781cfada6e980e5be1d69f
Sep 12 00:26:10.377549 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:10.377513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7c784bcf-w5sjw" event={"ID":"13f345ef-1e12-402d-9577-38b71e30e676","Type":"ContainerStarted","Data":"b63a0c95e91a31425f8711817de5ec7eb3301ed07907995211538eb48256a8de"}
Sep 12 00:26:10.377945 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:10.377564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7c784bcf-w5sjw" event={"ID":"13f345ef-1e12-402d-9577-38b71e30e676","Type":"ContainerStarted","Data":"973d31fb79837b59c29375db525db6b898d9e647b3781cfada6e980e5be1d69f"}
Sep 12 00:26:10.396870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:10.396823    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f7c784bcf-w5sjw" podStartSLOduration=1.396806658 podStartE2EDuration="1.396806658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:10.395250283 +0000 UTC m=+566.529240520" watchObservedRunningTime="2026-09-12 00:26:10.396806658 +0000 UTC m=+566.530796882"
Sep 12 00:26:13.341919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:13.341888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-qpsx4"
Sep 12 00:26:15.367884 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.367851    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-g78ps"]
Sep 12 00:26:15.372303 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.372283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.377300 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.377284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 12 00:26:15.379102 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.378901    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 12 00:26:15.379368 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.379351    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 12 00:26:15.379886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.379870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 12 00:26:15.380011 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.379996    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4h9x9\""
Sep 12 00:26:15.385617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.385593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-g78ps"]
Sep 12 00:26:15.439072 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.439036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9k4v\" (UniqueName: \"kubernetes.io/projected/501c7316-0153-4431-b6ce-0160608b9a0c-kube-api-access-s9k4v\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.439238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.439081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501c7316-0153-4431-b6ce-0160608b9a0c-tmp\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.439238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.439111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.539494 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.539456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9k4v\" (UniqueName: \"kubernetes.io/projected/501c7316-0153-4431-b6ce-0160608b9a0c-kube-api-access-s9k4v\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.539666 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.539504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501c7316-0153-4431-b6ce-0160608b9a0c-tmp\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.539666 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.539536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.539666 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:26:15.539628    2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 12 00:26:15.539841 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:26:15.539691    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs podName:501c7316-0153-4431-b6ce-0160608b9a0c nodeName:}" failed. No retries permitted until 2026-09-12 00:26:16.039672001 +0000 UTC m=+572.173662203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs") pod "kube-auth-proxy-76b8774589-g78ps" (UID: "501c7316-0153-4431-b6ce-0160608b9a0c") : secret "kube-auth-proxy-tls" not found
Sep 12 00:26:15.541716 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.541693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501c7316-0153-4431-b6ce-0160608b9a0c-tmp\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:15.552667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:15.552645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9k4v\" (UniqueName: \"kubernetes.io/projected/501c7316-0153-4431-b6ce-0160608b9a0c-kube-api-access-s9k4v\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:16.043399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:16.043348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:16.045683 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:16.045654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/501c7316-0153-4431-b6ce-0160608b9a0c-tls-certs\") pod \"kube-auth-proxy-76b8774589-g78ps\" (UID: \"501c7316-0153-4431-b6ce-0160608b9a0c\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:16.281706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:16.281665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps"
Sep 12 00:26:16.553585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:16.553557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-g78ps"]
Sep 12 00:26:16.556024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:26:16.555995    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501c7316_0153_4431_b6ce_0160608b9a0c.slice/crio-b0bba3056500e37bcbf748e26145052b5d8887634e163bb25c4ab23c62f7eb8b WatchSource:0}: Error finding container b0bba3056500e37bcbf748e26145052b5d8887634e163bb25c4ab23c62f7eb8b: Status 404 returned error can't find the container with id b0bba3056500e37bcbf748e26145052b5d8887634e163bb25c4ab23c62f7eb8b
Sep 12 00:26:17.411740 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:17.411700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps" event={"ID":"501c7316-0153-4431-b6ce-0160608b9a0c","Type":"ContainerStarted","Data":"b0bba3056500e37bcbf748e26145052b5d8887634e163bb25c4ab23c62f7eb8b"}
Sep 12 00:26:19.359423 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:19.359382    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:19.359813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:19.359447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:19.364476 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:19.364446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:19.425519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:19.425491    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f7c784bcf-w5sjw"
Sep 12 00:26:20.427728 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.427675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps" event={"ID":"501c7316-0153-4431-b6ce-0160608b9a0c","Type":"ContainerStarted","Data":"8e486959c6cb981a537dbf8a6b8b94b4a5e5bdd65672d8e2fa33841d9c14af3d"}
Sep 12 00:26:20.446602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.446549    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76b8774589-g78ps" podStartSLOduration=2.631436988 podStartE2EDuration="5.446513039s" totalImagesPullingTime="2.815076051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:20.445555461 +0000 UTC m=+576.579545687" watchObservedRunningTime="2026-09-12 00:26:20.446513039 +0000 UTC m=+576.580503264"
Sep 12 00:26:20.910824 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.910727    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"]
Sep 12 00:26:20.913992 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.913972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.916217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.916191    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 12 00:26:20.916347 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.916235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-76fbt\""
Sep 12 00:26:20.924859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.924834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"]
Sep 12 00:26:20.984675 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984675 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.984894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da45d3d9-52c8-4978-8e87-2b435cb10994-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.985066 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qww4t\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-kube-api-access-qww4t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:20.985066 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:20.984949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.085886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.085851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.085886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.085887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.085906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da45d3d9-52c8-4978-8e87-2b435cb10994-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qww4t\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-kube-api-access-qww4t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086497 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086497 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.086865 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.086843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da45d3d9-52c8-4978-8e87-2b435cb10994-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.088356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.088332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.088477 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.088463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.093301 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.093280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.093503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.093483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qww4t\" (UniqueName: \"kubernetes.io/projected/da45d3d9-52c8-4978-8e87-2b435cb10994-kube-api-access-qww4t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm\" (UID: \"da45d3d9-52c8-4978-8e87-2b435cb10994\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.224145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.224113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:21.498464 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:21.498440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"]
Sep 12 00:26:21.500530 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:26:21.500502    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda45d3d9_52c8_4978_8e87_2b435cb10994.slice/crio-f9a95ddf6ef97470cdb7cc17206be13f095f4bfc65a306751f705f24b12b8c45 WatchSource:0}: Error finding container f9a95ddf6ef97470cdb7cc17206be13f095f4bfc65a306751f705f24b12b8c45: Status 404 returned error can't find the container with id f9a95ddf6ef97470cdb7cc17206be13f095f4bfc65a306751f705f24b12b8c45
Sep 12 00:26:22.439598 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:22.439563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm" event={"ID":"da45d3d9-52c8-4978-8e87-2b435cb10994","Type":"ContainerStarted","Data":"f9a95ddf6ef97470cdb7cc17206be13f095f4bfc65a306751f705f24b12b8c45"}
Sep 12 00:26:24.190688 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:24.190652    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:26:24.190944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:24.190727    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:26:24.190944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:24.190758    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:26:24.450689 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:24.450602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm" event={"ID":"da45d3d9-52c8-4978-8e87-2b435cb10994","Type":"ContainerStarted","Data":"8302688e14c1632defbc8941ff3fd772a6858f31af60c35f0ed030290365d860"}
Sep 12 00:26:24.470890 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:24.470836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm" podStartSLOduration=1.783084028 podStartE2EDuration="4.470820745s" totalImagesPullingTime="2.687736717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:24.469313965 +0000 UTC m=+580.603304190" watchObservedRunningTime="2026-09-12 00:26:24.470820745 +0000 UTC m=+580.604810970"
Sep 12 00:26:25.224812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:25.224755    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:25.229190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:25.229159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:25.454893 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:25.454865    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:25.455742 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:25.455722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm"
Sep 12 00:26:26.052161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.052125    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m72zr"]
Sep 12 00:26:26.055506 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.055488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.057531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.057489    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 12 00:26:26.057666 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.057544    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qwq56\""
Sep 12 00:26:26.063042 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.062877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m72zr"]
Sep 12 00:26:26.128143 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.128112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e60429e-5b41-4503-b185-ed999848015e-cert\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.128143 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.128147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq25g\" (UniqueName: \"kubernetes.io/projected/1e60429e-5b41-4503-b185-ed999848015e-kube-api-access-kq25g\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.229264 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.229222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e60429e-5b41-4503-b185-ed999848015e-cert\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.229264 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.229268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq25g\" (UniqueName: \"kubernetes.io/projected/1e60429e-5b41-4503-b185-ed999848015e-kube-api-access-kq25g\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.231660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.231641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e60429e-5b41-4503-b185-ed999848015e-cert\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.236538 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.236521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq25g\" (UniqueName: \"kubernetes.io/projected/1e60429e-5b41-4503-b185-ed999848015e-kube-api-access-kq25g\") pod \"odh-model-controller-7998c67dc4-m72zr\" (UID: \"1e60429e-5b41-4503-b185-ed999848015e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.367140 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.367052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:26.637412 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:26.637333    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m72zr"]
Sep 12 00:26:26.640637 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:26:26.640608    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e60429e_5b41_4503_b185_ed999848015e.slice/crio-6709fa3a97d5d8e975e210dc4821f782f8ecf4921eb2ffffb0575bcab9e8fcdc WatchSource:0}: Error finding container 6709fa3a97d5d8e975e210dc4821f782f8ecf4921eb2ffffb0575bcab9e8fcdc: Status 404 returned error can't find the container with id 6709fa3a97d5d8e975e210dc4821f782f8ecf4921eb2ffffb0575bcab9e8fcdc
Sep 12 00:26:27.467484 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:27.467446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr" event={"ID":"1e60429e-5b41-4503-b185-ed999848015e","Type":"ContainerStarted","Data":"6709fa3a97d5d8e975e210dc4821f782f8ecf4921eb2ffffb0575bcab9e8fcdc"}
Sep 12 00:26:30.482183 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:30.482143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr" event={"ID":"1e60429e-5b41-4503-b185-ed999848015e","Type":"ContainerStarted","Data":"a5b276470279079d81857259bed0528c1b937b4346edf89da6db39f5736bca6d"}
Sep 12 00:26:30.482611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:30.482401    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:30.499965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:30.499911    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr" podStartSLOduration=1.604628903 podStartE2EDuration="4.499895892s" totalImagesPullingTime="2.895266989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:30.497714159 +0000 UTC m=+586.631704409" watchObservedRunningTime="2026-09-12 00:26:30.499895892 +0000 UTC m=+586.633886114"
Sep 12 00:26:33.070225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.070167    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"]
Sep 12 00:26:33.078355 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.078325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.081511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.081483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 12 00:26:33.081511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.081497    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 12 00:26:33.082324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.082299    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 12 00:26:33.082443 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.082350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 12 00:26:33.082443 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.082367    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-28vf9\""
Sep 12 00:26:33.082642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.082626    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:26:33.083618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.083594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"]
Sep 12 00:26:33.191821 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.191784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcsmw\" (UniqueName: \"kubernetes.io/projected/15a1e0cb-4c11-41cd-884d-0c1b9535726d-kube-api-access-pcsmw\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.191978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.191829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15a1e0cb-4c11-41cd-884d-0c1b9535726d-manager-config\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.191978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.191928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.191978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.191959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-metrics-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.293032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.292995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.293032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.293031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-metrics-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.293227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.293072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcsmw\" (UniqueName: \"kubernetes.io/projected/15a1e0cb-4c11-41cd-884d-0c1b9535726d-kube-api-access-pcsmw\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.293227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.293099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15a1e0cb-4c11-41cd-884d-0c1b9535726d-manager-config\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.293885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.293860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15a1e0cb-4c11-41cd-884d-0c1b9535726d-manager-config\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.295676 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.295643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-metrics-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.295676 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.295660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15a1e0cb-4c11-41cd-884d-0c1b9535726d-cert\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.301679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.301660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcsmw\" (UniqueName: \"kubernetes.io/projected/15a1e0cb-4c11-41cd-884d-0c1b9535726d-kube-api-access-pcsmw\") pod \"lws-controller-manager-7b5cf77b9-vxqkk\" (UID: \"15a1e0cb-4c11-41cd-884d-0c1b9535726d\") " pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.389380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.389314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:33.662671 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:33.662593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"]
Sep 12 00:26:33.665725 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:26:33.665699    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a1e0cb_4c11_41cd_884d_0c1b9535726d.slice/crio-ac6ec81f48ff8dfb87ee50497ea0f95205ecf7cf482dfc1e9b9e5b12e0019601 WatchSource:0}: Error finding container ac6ec81f48ff8dfb87ee50497ea0f95205ecf7cf482dfc1e9b9e5b12e0019601: Status 404 returned error can't find the container with id ac6ec81f48ff8dfb87ee50497ea0f95205ecf7cf482dfc1e9b9e5b12e0019601
Sep 12 00:26:34.503179 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:34.503099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk" event={"ID":"15a1e0cb-4c11-41cd-884d-0c1b9535726d","Type":"ContainerStarted","Data":"ac6ec81f48ff8dfb87ee50497ea0f95205ecf7cf482dfc1e9b9e5b12e0019601"}
Sep 12 00:26:35.509986 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:35.509947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk" event={"ID":"15a1e0cb-4c11-41cd-884d-0c1b9535726d","Type":"ContainerStarted","Data":"f2e6fd6cef7dafeee59052db551408b3660adb77aa51c1a9de58e552600decd6"}
Sep 12 00:26:35.510423 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:35.510025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:26:35.526218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:35.526162    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk" podStartSLOduration=1.182489177 podStartE2EDuration="2.526146293s" totalImagesPullingTime="1.343657116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:35.525370468 +0000 UTC m=+591.659360694" watchObservedRunningTime="2026-09-12 00:26:35.526146293 +0000 UTC m=+591.660136517"
Sep 12 00:26:41.488186 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:41.488154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-m72zr"
Sep 12 00:26:46.517684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:26:46.517654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b5cf77b9-vxqkk"
Sep 12 00:27:30.684953 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.684918    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"]
Sep 12 00:27:30.688404 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.688385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.690522 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.690502    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 00:27:30.691183 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.691161    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 00:27:30.691304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.691169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ft8qr\""
Sep 12 00:27:30.696315 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.696225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"]
Sep 12 00:27:30.779108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.779073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmnk7\" (UniqueName: \"kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.779279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.779130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.779279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.779156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.788502 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.788475    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"]
Sep 12 00:27:30.791807 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.791791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.799383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.799354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"]
Sep 12 00:27:30.879663 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmnk7\" (UniqueName: \"kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.879870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.879870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.879870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.879870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgtx\" (UniqueName: \"kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.879870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.879815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.880105 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.880087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.880141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.880129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.887279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.887258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmnk7\" (UniqueName: \"kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:30.890734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.890715    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"]
Sep 12 00:27:30.894182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.894166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:30.902198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.902177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"]
Sep 12 00:27:30.981228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:30.981397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.981397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.981397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:30.981397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbktq\" (UniqueName: \"kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:30.981630 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgtx\" (UniqueName: \"kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.981630 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.981706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.981657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.989988 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.989963    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"]
Sep 12 00:27:30.992078 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.992057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgtx\" (UniqueName: \"kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:30.993393 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.993376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:30.998886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:30.998861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:31.000586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.000567    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"]
Sep 12 00:27:31.082352 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bjbh\" (UniqueName: \"kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.082492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.082492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.082590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.082628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.082663 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbktq\" (UniqueName: \"kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.082759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082745    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.082934 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.082917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.090333 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.090299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbktq\" (UniqueName: \"kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.100918 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.100901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:31.183971 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.183943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.184125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.183999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bjbh\" (UniqueName: \"kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.184125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.184040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.184384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.184362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.184425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.184385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.192735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.192711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bjbh\" (UniqueName: \"kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.203678 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.203662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:31.272974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.272947    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"]
Sep 12 00:27:31.275483 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:27:31.275443    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3810136_4d37_4988_b56b_d66fa1adc2ad.slice/crio-ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f WatchSource:0}: Error finding container ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f: Status 404 returned error can't find the container with id ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f
Sep 12 00:27:31.308543 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.308521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:31.379786 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.376208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"]
Sep 12 00:27:31.434659 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:27:31.434621    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abcf4ac_d182_4115_8a58_81ee21a290c4.slice/crio-a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d WatchSource:0}: Error finding container a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d: Status 404 returned error can't find the container with id a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d
Sep 12 00:27:31.474006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.473985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"]
Sep 12 00:27:31.475425 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:27:31.475402    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbc9740_fd94_452e_a4e7_f4ed0fb22e73.slice/crio-de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e WatchSource:0}: Error finding container de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e: Status 404 returned error can't find the container with id de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e
Sep 12 00:27:31.581882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.581859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"]
Sep 12 00:27:31.644626 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:27:31.644594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eae64e3_df75_4d67_9dbd_9b997b2e9a5d.slice/crio-35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3 WatchSource:0}: Error finding container 35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3: Status 404 returned error can't find the container with id 35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3
Sep 12 00:27:31.770003 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.769924    2580 generic.go:356] "Generic (PLEG): container finished" podID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerID="2ca535317c6ba73ac936723869865276c2d92a17aabe426fbd8edcc9ce89a20b" exitCode=0
Sep 12 00:27:31.770391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.770002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q" event={"ID":"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73","Type":"ContainerDied","Data":"2ca535317c6ba73ac936723869865276c2d92a17aabe426fbd8edcc9ce89a20b"}
Sep 12 00:27:31.770391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.770064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q" event={"ID":"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73","Type":"ContainerStarted","Data":"de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e"}
Sep 12 00:27:31.771581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.771563    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerID="9275190699a6920ee39323c0d3a60cf4df11d38587358ea60c0fea402598630f" exitCode=0
Sep 12 00:27:31.771670 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.771647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q" event={"ID":"a3810136-4d37-4988-b56b-d66fa1adc2ad","Type":"ContainerDied","Data":"9275190699a6920ee39323c0d3a60cf4df11d38587358ea60c0fea402598630f"}
Sep 12 00:27:31.771736 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.771692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q" event={"ID":"a3810136-4d37-4988-b56b-d66fa1adc2ad","Type":"ContainerStarted","Data":"ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f"}
Sep 12 00:27:31.773283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.773261    2580 generic.go:356] "Generic (PLEG): container finished" podID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerID="b3e4ef9993290ccc14fea1c8d2fd1f07ba77e3832890b6702469c11e0ec7fa66" exitCode=0
Sep 12 00:27:31.773504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.773344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r" event={"ID":"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d","Type":"ContainerDied","Data":"b3e4ef9993290ccc14fea1c8d2fd1f07ba77e3832890b6702469c11e0ec7fa66"}
Sep 12 00:27:31.773504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.773388    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r" event={"ID":"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d","Type":"ContainerStarted","Data":"35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3"}
Sep 12 00:27:31.775141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.775115    2580 generic.go:356] "Generic (PLEG): container finished" podID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerID="5eec4e9e14182436c445f7f4e17a5d8a70517194c15a4c2572d859354abbffd3" exitCode=0
Sep 12 00:27:31.775202 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.775170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4" event={"ID":"6abcf4ac-d182-4115-8a58-81ee21a290c4","Type":"ContainerDied","Data":"5eec4e9e14182436c445f7f4e17a5d8a70517194c15a4c2572d859354abbffd3"}
Sep 12 00:27:31.775262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:31.775204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4" event={"ID":"6abcf4ac-d182-4115-8a58-81ee21a290c4","Type":"ContainerStarted","Data":"a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d"}
Sep 12 00:27:32.781567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:32.781487    2580 generic.go:356] "Generic (PLEG): container finished" podID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerID="0caaa0f6472f2d7fbf5d0d0eb731978fc63844b9030871ff0900a17fb4f9fcff" exitCode=0
Sep 12 00:27:32.781567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:32.781541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q" event={"ID":"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73","Type":"ContainerDied","Data":"0caaa0f6472f2d7fbf5d0d0eb731978fc63844b9030871ff0900a17fb4f9fcff"}
Sep 12 00:27:33.787832 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.787800    2580 generic.go:356] "Generic (PLEG): container finished" podID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerID="acff784c0c6a9e085607832842f2a956076ccbc38a04941f63ad3a3e843a4802" exitCode=0
Sep 12 00:27:33.788314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.787889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r" event={"ID":"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d","Type":"ContainerDied","Data":"acff784c0c6a9e085607832842f2a956076ccbc38a04941f63ad3a3e843a4802"}
Sep 12 00:27:33.789877 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.789859    2580 generic.go:356] "Generic (PLEG): container finished" podID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerID="414377fcec38ee8795d4a0eea07bcfc2b77beecb0e54adcbf3d65afbbdf7bb36" exitCode=0
Sep 12 00:27:33.789968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.789943    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4" event={"ID":"6abcf4ac-d182-4115-8a58-81ee21a290c4","Type":"ContainerDied","Data":"414377fcec38ee8795d4a0eea07bcfc2b77beecb0e54adcbf3d65afbbdf7bb36"}
Sep 12 00:27:33.792062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.792041    2580 generic.go:356] "Generic (PLEG): container finished" podID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerID="ab2c7c45d9bf56f4035442348c977c29d63c6da861f30042cd5b14b883de6e95" exitCode=0
Sep 12 00:27:33.792163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.792113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q" event={"ID":"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73","Type":"ContainerDied","Data":"ab2c7c45d9bf56f4035442348c977c29d63c6da861f30042cd5b14b883de6e95"}
Sep 12 00:27:33.793777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.793744    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerID="996a2b6efad85360f1931701c9cf3f9943ffb49bb64cd73aeb67ed80a40f4b2f" exitCode=0
Sep 12 00:27:33.793864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:33.793799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q" event={"ID":"a3810136-4d37-4988-b56b-d66fa1adc2ad","Type":"ContainerDied","Data":"996a2b6efad85360f1931701c9cf3f9943ffb49bb64cd73aeb67ed80a40f4b2f"}
Sep 12 00:27:34.803209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.803175    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerID="2f3d78b74d3870bc249e20f9ed4147d2723e3813d9834dfe9f2d296bbf673792" exitCode=0
Sep 12 00:27:34.803581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.803258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q" event={"ID":"a3810136-4d37-4988-b56b-d66fa1adc2ad","Type":"ContainerDied","Data":"2f3d78b74d3870bc249e20f9ed4147d2723e3813d9834dfe9f2d296bbf673792"}
Sep 12 00:27:34.805287 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.805264    2580 generic.go:356] "Generic (PLEG): container finished" podID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerID="3459bf62b7a3512f574b34f5f2da8c4bcf86b22b56456b5d03edc47b649f8061" exitCode=0
Sep 12 00:27:34.805373 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.805352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r" event={"ID":"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d","Type":"ContainerDied","Data":"3459bf62b7a3512f574b34f5f2da8c4bcf86b22b56456b5d03edc47b649f8061"}
Sep 12 00:27:34.807378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.807352    2580 generic.go:356] "Generic (PLEG): container finished" podID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerID="e4b1ead1aa097e53d9c5aab3133a0c4a8f9be84991fea29453ae012d089ce54e" exitCode=0
Sep 12 00:27:34.807487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.807405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4" event={"ID":"6abcf4ac-d182-4115-8a58-81ee21a290c4","Type":"ContainerDied","Data":"e4b1ead1aa097e53d9c5aab3133a0c4a8f9be84991fea29453ae012d089ce54e"}
Sep 12 00:27:34.934806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:34.934780    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:35.117091 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.117002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle\") pod \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") "
Sep 12 00:27:35.117260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.117096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbktq\" (UniqueName: \"kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq\") pod \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") "
Sep 12 00:27:35.117260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.117124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util\") pod \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\" (UID: \"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73\") "
Sep 12 00:27:35.117647 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.117614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle" (OuterVolumeSpecName: "bundle") pod "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" (UID: "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:35.119214 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.119192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq" (OuterVolumeSpecName: "kube-api-access-zbktq") pod "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" (UID: "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73"). InnerVolumeSpecName "kube-api-access-zbktq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:27:35.122446 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.122424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util" (OuterVolumeSpecName: "util") pod "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" (UID: "7bbc9740-fd94-452e-a4e7-f4ed0fb22e73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:35.218511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.218456    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:35.218511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.218504    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:35.218511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.218514    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbktq\" (UniqueName: \"kubernetes.io/projected/7bbc9740-fd94-452e-a4e7-f4ed0fb22e73-kube-api-access-zbktq\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:35.814655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.814627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q"
Sep 12 00:27:35.815098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.814658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccwm5q" event={"ID":"7bbc9740-fd94-452e-a4e7-f4ed0fb22e73","Type":"ContainerDied","Data":"de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e"}
Sep 12 00:27:35.815098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.814699    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de757a72a7d68d438fd05d9b26e15089fea696d8b7f6011fb8ff9f1924a1b92e"
Sep 12 00:27:35.960449 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.960422    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:35.995510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.995486    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:35.998756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:35.998728    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:36.125063 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.124963    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util\") pod \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") "
Sep 12 00:27:36.125063 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125053    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmnk7\" (UniqueName: \"kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7\") pod \"a3810136-4d37-4988-b56b-d66fa1adc2ad\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfgtx\" (UniqueName: \"kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx\") pod \"6abcf4ac-d182-4115-8a58-81ee21a290c4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle\") pod \"6abcf4ac-d182-4115-8a58-81ee21a290c4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util\") pod \"6abcf4ac-d182-4115-8a58-81ee21a290c4\" (UID: \"6abcf4ac-d182-4115-8a58-81ee21a290c4\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle\") pod \"a3810136-4d37-4988-b56b-d66fa1adc2ad\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125222    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util\") pod \"a3810136-4d37-4988-b56b-d66fa1adc2ad\" (UID: \"a3810136-4d37-4988-b56b-d66fa1adc2ad\") "
Sep 12 00:27:36.125294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bjbh\" (UniqueName: \"kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh\") pod \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") "
Sep 12 00:27:36.125569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125295    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle\") pod \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\" (UID: \"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d\") "
Sep 12 00:27:36.125959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125921    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle" (OuterVolumeSpecName: "bundle") pod "6abcf4ac-d182-4115-8a58-81ee21a290c4" (UID: "6abcf4ac-d182-4115-8a58-81ee21a290c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.126062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.125969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle" (OuterVolumeSpecName: "bundle") pod "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" (UID: "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.126150 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.126125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle" (OuterVolumeSpecName: "bundle") pod "a3810136-4d37-4988-b56b-d66fa1adc2ad" (UID: "a3810136-4d37-4988-b56b-d66fa1adc2ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.127826 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.127797    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx" (OuterVolumeSpecName: "kube-api-access-wfgtx") pod "6abcf4ac-d182-4115-8a58-81ee21a290c4" (UID: "6abcf4ac-d182-4115-8a58-81ee21a290c4"). InnerVolumeSpecName "kube-api-access-wfgtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:27:36.127974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.127948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh" (OuterVolumeSpecName: "kube-api-access-5bjbh") pod "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" (UID: "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d"). InnerVolumeSpecName "kube-api-access-5bjbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:27:36.128170 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.128150    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7" (OuterVolumeSpecName: "kube-api-access-zmnk7") pod "a3810136-4d37-4988-b56b-d66fa1adc2ad" (UID: "a3810136-4d37-4988-b56b-d66fa1adc2ad"). InnerVolumeSpecName "kube-api-access-zmnk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:27:36.131371 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.131339    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util" (OuterVolumeSpecName: "util") pod "6abcf4ac-d182-4115-8a58-81ee21a290c4" (UID: "6abcf4ac-d182-4115-8a58-81ee21a290c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.133560 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.133531    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util" (OuterVolumeSpecName: "util") pod "a3810136-4d37-4988-b56b-d66fa1adc2ad" (UID: "a3810136-4d37-4988-b56b-d66fa1adc2ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.133885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.133868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util" (OuterVolumeSpecName: "util") pod "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" (UID: "6eae64e3-df75-4d67-9dbd-9b997b2e9a5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:27:36.226090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226048    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226081    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3810136-4d37-4988-b56b-d66fa1adc2ad-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226094    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bjbh\" (UniqueName: \"kubernetes.io/projected/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-kube-api-access-5bjbh\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226107    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226119    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6eae64e3-df75-4d67-9dbd-9b997b2e9a5d-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226129    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmnk7\" (UniqueName: \"kubernetes.io/projected/a3810136-4d37-4988-b56b-d66fa1adc2ad-kube-api-access-zmnk7\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226142    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfgtx\" (UniqueName: \"kubernetes.io/projected/6abcf4ac-d182-4115-8a58-81ee21a290c4-kube-api-access-wfgtx\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226153    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-bundle\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.226374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.226163    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6abcf4ac-d182-4115-8a58-81ee21a290c4-util\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:27:36.821425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.821397    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r"
Sep 12 00:27:36.821869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.821396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c664t2r" event={"ID":"6eae64e3-df75-4d67-9dbd-9b997b2e9a5d","Type":"ContainerDied","Data":"35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3"}
Sep 12 00:27:36.821869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.821554    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35fa558950e8b7a74dd775661a6ad2bb2f414522f83083d84c8d5e80dcf6cbb3"
Sep 12 00:27:36.823411 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.823388    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4"
Sep 12 00:27:36.823567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.823394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7sqhf4" event={"ID":"6abcf4ac-d182-4115-8a58-81ee21a290c4","Type":"ContainerDied","Data":"a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d"}
Sep 12 00:27:36.823567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.823532    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a212ce11e547e78958c313d6c105cdb58d0bdc4132df56a83dd6b48ed525c65d"
Sep 12 00:27:36.825601 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.825583    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q"
Sep 12 00:27:36.825701 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.825607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847assq9q" event={"ID":"a3810136-4d37-4988-b56b-d66fa1adc2ad","Type":"ContainerDied","Data":"ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f"}
Sep 12 00:27:36.825701 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:27:36.825646    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad9c33971e44351093f4e8576c2cce49d02d207b44f981b9b4ac7ef0c2cd0b6f"
Sep 12 00:28:57.147968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.147930    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148224    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148237    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148256    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148264    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148274    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148282    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148293    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148299    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148305    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148310    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148316    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148321    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148329    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148336    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148353    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148361    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148374    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148382    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="util"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148392    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148399    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148409    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148418    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148427    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148435    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="pull"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148511    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eae64e3-df75-4d67-9dbd-9b997b2e9a5d" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148523    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3810136-4d37-4988-b56b-d66fa1adc2ad" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148535    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6abcf4ac-d182-4115-8a58-81ee21a290c4" containerName="extract"
Sep 12 00:28:57.148536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.148546    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bbc9740-fd94-452e-a4e7-f4ed0fb22e73" containerName="extract"
Sep 12 00:28:57.151396 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.151378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:28:57.153530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.153507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 12 00:28:57.153644 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.153513    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tt7ln\""
Sep 12 00:28:57.154291 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.154274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 12 00:28:57.157428 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.157404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:28:57.215808 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.215757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5d5s\" (UniqueName: \"kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s\") pod \"authorino-7bcfd8f9cd-wbh5f\" (UID: \"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:28:57.317204 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.317168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5d5s\" (UniqueName: \"kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s\") pod \"authorino-7bcfd8f9cd-wbh5f\" (UID: \"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:28:57.324774 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.324746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5d5s\" (UniqueName: \"kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s\") pod \"authorino-7bcfd8f9cd-wbh5f\" (UID: \"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:28:57.461079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.461037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:28:57.726871 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:57.726792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:28:57.730482 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:28:57.730442    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd63e4b01_440d_41d9_b1b5_3e5ab62f0fd5.slice/crio-889fb199f0a2d4895cb9865ac43a80c3996df8bea490acfa9fb6fd7c2f20e116 WatchSource:0}: Error finding container 889fb199f0a2d4895cb9865ac43a80c3996df8bea490acfa9fb6fd7c2f20e116: Status 404 returned error can't find the container with id 889fb199f0a2d4895cb9865ac43a80c3996df8bea490acfa9fb6fd7c2f20e116
Sep 12 00:28:58.209957 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:58.209925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" event={"ID":"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5","Type":"ContainerStarted","Data":"889fb199f0a2d4895cb9865ac43a80c3996df8bea490acfa9fb6fd7c2f20e116"}
Sep 12 00:28:59.637046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:28:59.637005    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:29:01.227281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.227236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" event={"ID":"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5","Type":"ContainerStarted","Data":"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"}
Sep 12 00:29:01.227711 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.227301    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" podUID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" containerName="authorino" containerID="cri-o://a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a" gracePeriod=30
Sep 12 00:29:01.241661 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.241618    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" podStartSLOduration=1.802388481 podStartE2EDuration="4.241602233s" totalImagesPullingTime="2.439213752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:01.240316173 +0000 UTC m=+737.374306399" watchObservedRunningTime="2026-09-12 00:29:01.241602233 +0000 UTC m=+737.375592459"
Sep 12 00:29:01.471085 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.471061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:29:01.554183 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.554094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5d5s\" (UniqueName: \"kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s\") pod \"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5\" (UID: \"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5\") "
Sep 12 00:29:01.556183 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.556160    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s" (OuterVolumeSpecName: "kube-api-access-m5d5s") pod "d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" (UID: "d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5"). InnerVolumeSpecName "kube-api-access-m5d5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:29:01.655325 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:01.655283    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5d5s\" (UniqueName: \"kubernetes.io/projected/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5-kube-api-access-m5d5s\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:29:02.233938 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.233902    2580 generic.go:356] "Generic (PLEG): container finished" podID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" containerID="a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a" exitCode=0
Sep 12 00:29:02.234402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.233944    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f"
Sep 12 00:29:02.234402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.233962    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" event={"ID":"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5","Type":"ContainerDied","Data":"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"}
Sep 12 00:29:02.234402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.233994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-wbh5f" event={"ID":"d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5","Type":"ContainerDied","Data":"889fb199f0a2d4895cb9865ac43a80c3996df8bea490acfa9fb6fd7c2f20e116"}
Sep 12 00:29:02.234402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.234018    2580 scope.go:117] "RemoveContainer" containerID="a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"
Sep 12 00:29:02.243052 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.242932    2580 scope.go:117] "RemoveContainer" containerID="a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"
Sep 12 00:29:02.243283 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:29:02.243256    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a\": container with ID starting with a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a not found: ID does not exist" containerID="a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"
Sep 12 00:29:02.243342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.243292    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a"} err="failed to get container status \"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a\": rpc error: code = NotFound desc = could not find container \"a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a\": container with ID starting with a1a816cbe7eb9d600b08c9739da2d499d04b361d7b5bc03521b709da5bee7b6a not found: ID does not exist"
Sep 12 00:29:02.263586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.263551    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:29:02.273656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.273632    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-wbh5f"]
Sep 12 00:29:02.373457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:02.373426    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" path="/var/lib/kubelet/pods/d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5/volumes"
Sep 12 00:29:27.783662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.783624    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:27.784121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.783974    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" containerName="authorino"
Sep 12 00:29:27.784121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.783988    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" containerName="authorino"
Sep 12 00:29:27.784121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.784045    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63e4b01-440d-41d9-b1b5-3e5ab62f0fd5" containerName="authorino"
Sep 12 00:29:27.791954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.791920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:27.793385 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.793359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:27.794296 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.794273    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tt7ln\""
Sep 12 00:29:27.794483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.794314    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 12 00:29:27.795065 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.795047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 12 00:29:27.884918 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.884881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47vbl\" (UniqueName: \"kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl\") pod \"authorino-557f876f69-lqqrq\" (UID: \"2e163a1c-ee58-4d93-abf2-274bfc1c2612\") " pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:27.985464 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.985431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47vbl\" (UniqueName: \"kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl\") pod \"authorino-557f876f69-lqqrq\" (UID: \"2e163a1c-ee58-4d93-abf2-274bfc1c2612\") " pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:27.993682 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:27.993650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47vbl\" (UniqueName: \"kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl\") pod \"authorino-557f876f69-lqqrq\" (UID: \"2e163a1c-ee58-4d93-abf2-274bfc1c2612\") " pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:28.023953 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.023920    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:28.024184 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.024171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:28.050657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.050628    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:28.054523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.054505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:28.060331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.060305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:28.186856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.186813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bn2v\" (UniqueName: \"kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v\") pod \"authorino-86966dc6c4-rd5gp\" (UID: \"58d916f1-c81c-460b-8c1d-cce97ddb8814\") " pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:28.287449 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.287380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bn2v\" (UniqueName: \"kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v\") pod \"authorino-86966dc6c4-rd5gp\" (UID: \"58d916f1-c81c-460b-8c1d-cce97ddb8814\") " pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:28.296655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.296591    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:28.297924 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.297901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bn2v\" (UniqueName: \"kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v\") pod \"authorino-86966dc6c4-rd5gp\" (UID: \"58d916f1-c81c-460b-8c1d-cce97ddb8814\") " pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:28.298852 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:29:28.298825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e163a1c_ee58_4d93_abf2_274bfc1c2612.slice/crio-e916a680757859fe20b4f6c3f36afbb7cbce49d0e32bb90b132e2dcf026645da WatchSource:0}: Error finding container e916a680757859fe20b4f6c3f36afbb7cbce49d0e32bb90b132e2dcf026645da: Status 404 returned error can't find the container with id e916a680757859fe20b4f6c3f36afbb7cbce49d0e32bb90b132e2dcf026645da
Sep 12 00:29:28.364568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.364537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-lqqrq" event={"ID":"2e163a1c-ee58-4d93-abf2-274bfc1c2612","Type":"ContainerStarted","Data":"e916a680757859fe20b4f6c3f36afbb7cbce49d0e32bb90b132e2dcf026645da"}
Sep 12 00:29:28.366773 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.366747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:28.633814 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:28.633584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:28.635848 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:29:28.635810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58d916f1_c81c_460b_8c1d_cce97ddb8814.slice/crio-b3ec7850086ff7f234032d3fdf1e7ad0cf463f41b916d4f8bce4e00097dc36b6 WatchSource:0}: Error finding container b3ec7850086ff7f234032d3fdf1e7ad0cf463f41b916d4f8bce4e00097dc36b6: Status 404 returned error can't find the container with id b3ec7850086ff7f234032d3fdf1e7ad0cf463f41b916d4f8bce4e00097dc36b6
Sep 12 00:29:29.371472 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.371380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" event={"ID":"58d916f1-c81c-460b-8c1d-cce97ddb8814","Type":"ContainerStarted","Data":"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"}
Sep 12 00:29:29.371472 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.371423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" event={"ID":"58d916f1-c81c-460b-8c1d-cce97ddb8814","Type":"ContainerStarted","Data":"b3ec7850086ff7f234032d3fdf1e7ad0cf463f41b916d4f8bce4e00097dc36b6"}
Sep 12 00:29:29.372966 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.372942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-lqqrq" event={"ID":"2e163a1c-ee58-4d93-abf2-274bfc1c2612","Type":"ContainerStarted","Data":"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"}
Sep 12 00:29:29.373066 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.372983    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-lqqrq" podUID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" containerName="authorino" containerID="cri-o://db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7" gracePeriod=30
Sep 12 00:29:29.385539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.385391    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" podStartSLOduration=1.030396175 podStartE2EDuration="1.385376689s" totalImagesPullingTime="354.980514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:29.383977891 +0000 UTC m=+765.517968116" watchObservedRunningTime="2026-09-12 00:29:29.385376689 +0000 UTC m=+765.519366914"
Sep 12 00:29:29.398335 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.398294    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-lqqrq" podStartSLOduration=1.906894625 podStartE2EDuration="2.398278955s" totalImagesPullingTime="491.38433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:29.396106393 +0000 UTC m=+765.530096619" watchObservedRunningTime="2026-09-12 00:29:29.398278955 +0000 UTC m=+765.532269209"
Sep 12 00:29:29.483477 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.483443    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:29:29.486665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.486647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.488587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.488568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 12 00:29:29.493204 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.493182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:29:29.598510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.598480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw2bb\" (UniqueName: \"kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.598655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.598545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.618888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.618866    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:29.698974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.698939    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47vbl\" (UniqueName: \"kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl\") pod \"2e163a1c-ee58-4d93-abf2-274bfc1c2612\" (UID: \"2e163a1c-ee58-4d93-abf2-274bfc1c2612\") "
Sep 12 00:29:29.699144 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.699050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw2bb\" (UniqueName: \"kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.699144 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.699120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.701015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.700982    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl" (OuterVolumeSpecName: "kube-api-access-47vbl") pod "2e163a1c-ee58-4d93-abf2-274bfc1c2612" (UID: "2e163a1c-ee58-4d93-abf2-274bfc1c2612"). InnerVolumeSpecName "kube-api-access-47vbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:29:29.701402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.701385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.706152 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.706132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw2bb\" (UniqueName: \"kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb\") pod \"authorino-b6c9b9788-sqnnc\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") " pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.796498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.796459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:29:29.799563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:29.799542    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47vbl\" (UniqueName: \"kubernetes.io/projected/2e163a1c-ee58-4d93-abf2-274bfc1c2612-kube-api-access-47vbl\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:29:30.071366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.071325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:29:30.073684 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:29:30.073655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0dd5cc6_700e_4b35_95da_e7297592063a.slice/crio-0c73a27125296e7b1a4b70b809e2da616c42e0d61ef6cb5150aef8444703f8cf WatchSource:0}: Error finding container 0c73a27125296e7b1a4b70b809e2da616c42e0d61ef6cb5150aef8444703f8cf: Status 404 returned error can't find the container with id 0c73a27125296e7b1a4b70b809e2da616c42e0d61ef6cb5150aef8444703f8cf
Sep 12 00:29:30.379257 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.379176    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" containerID="db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7" exitCode=0
Sep 12 00:29:30.379257 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.379212    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-lqqrq"
Sep 12 00:29:30.379679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.379209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-lqqrq" event={"ID":"2e163a1c-ee58-4d93-abf2-274bfc1c2612","Type":"ContainerDied","Data":"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"}
Sep 12 00:29:30.379679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.379317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-lqqrq" event={"ID":"2e163a1c-ee58-4d93-abf2-274bfc1c2612","Type":"ContainerDied","Data":"e916a680757859fe20b4f6c3f36afbb7cbce49d0e32bb90b132e2dcf026645da"}
Sep 12 00:29:30.379679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.379342    2580 scope.go:117] "RemoveContainer" containerID="db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"
Sep 12 00:29:30.381527 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.381492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" event={"ID":"d0dd5cc6-700e-4b35-95da-e7297592063a","Type":"ContainerStarted","Data":"0c73a27125296e7b1a4b70b809e2da616c42e0d61ef6cb5150aef8444703f8cf"}
Sep 12 00:29:30.394527 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.394506    2580 scope.go:117] "RemoveContainer" containerID="db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"
Sep 12 00:29:30.394816 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:29:30.394795    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7\": container with ID starting with db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7 not found: ID does not exist" containerID="db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"
Sep 12 00:29:30.394874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.394825    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7"} err="failed to get container status \"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7\": rpc error: code = NotFound desc = could not find container \"db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7\": container with ID starting with db281ba3c01d5e3142568d5886ad50b8db788a0d23b9d47043d3654e356d4ce7 not found: ID does not exist"
Sep 12 00:29:30.402862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.402841    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:30.406688 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:30.406666    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-lqqrq"]
Sep 12 00:29:31.387726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.387688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" event={"ID":"d0dd5cc6-700e-4b35-95da-e7297592063a","Type":"ContainerStarted","Data":"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"}
Sep 12 00:29:31.402866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.402822    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" podStartSLOduration=1.986802837 podStartE2EDuration="2.402807217s" totalImagesPullingTime="416.00438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:31.400548541 +0000 UTC m=+767.534538765" watchObservedRunningTime="2026-09-12 00:29:31.402807217 +0000 UTC m=+767.536797441"
Sep 12 00:29:31.428437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.428409    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:31.428590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.428571    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" podUID="58d916f1-c81c-460b-8c1d-cce97ddb8814" containerName="authorino" containerID="cri-o://5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511" gracePeriod=30
Sep 12 00:29:31.676405 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.676379    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:31.715776 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.715732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bn2v\" (UniqueName: \"kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v\") pod \"58d916f1-c81c-460b-8c1d-cce97ddb8814\" (UID: \"58d916f1-c81c-460b-8c1d-cce97ddb8814\") "
Sep 12 00:29:31.717884 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.717862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v" (OuterVolumeSpecName: "kube-api-access-4bn2v") pod "58d916f1-c81c-460b-8c1d-cce97ddb8814" (UID: "58d916f1-c81c-460b-8c1d-cce97ddb8814"). InnerVolumeSpecName "kube-api-access-4bn2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:29:31.817002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:31.816961    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bn2v\" (UniqueName: \"kubernetes.io/projected/58d916f1-c81c-460b-8c1d-cce97ddb8814-kube-api-access-4bn2v\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:29:32.373948 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.373910    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" path="/var/lib/kubelet/pods/2e163a1c-ee58-4d93-abf2-274bfc1c2612/volumes"
Sep 12 00:29:32.393723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.393694    2580 generic.go:356] "Generic (PLEG): container finished" podID="58d916f1-c81c-460b-8c1d-cce97ddb8814" containerID="5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511" exitCode=0
Sep 12 00:29:32.394118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.393733    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86966dc6c4-rd5gp"
Sep 12 00:29:32.394118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.393790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" event={"ID":"58d916f1-c81c-460b-8c1d-cce97ddb8814","Type":"ContainerDied","Data":"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"}
Sep 12 00:29:32.394118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.393839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86966dc6c4-rd5gp" event={"ID":"58d916f1-c81c-460b-8c1d-cce97ddb8814","Type":"ContainerDied","Data":"b3ec7850086ff7f234032d3fdf1e7ad0cf463f41b916d4f8bce4e00097dc36b6"}
Sep 12 00:29:32.394118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.393866    2580 scope.go:117] "RemoveContainer" containerID="5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"
Sep 12 00:29:32.402084 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.402069    2580 scope.go:117] "RemoveContainer" containerID="5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"
Sep 12 00:29:32.402326 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:29:32.402309    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511\": container with ID starting with 5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511 not found: ID does not exist" containerID="5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"
Sep 12 00:29:32.402378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.402333    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511"} err="failed to get container status \"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511\": rpc error: code = NotFound desc = could not find container \"5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511\": container with ID starting with 5aa7223b0723f8a1ebbc9be628e71b9395ccab824c9222dcafc52791918f7511 not found: ID does not exist"
Sep 12 00:29:32.408269 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.408248    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:32.411489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:32.411465    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86966dc6c4-rd5gp"]
Sep 12 00:29:34.374550 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:29:34.374518    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d916f1-c81c-460b-8c1d-cce97ddb8814" path="/var/lib/kubelet/pods/58d916f1-c81c-460b-8c1d-cce97ddb8814/volumes"
Sep 12 00:30:42.814709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.814675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"]
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.814987    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d916f1-c81c-460b-8c1d-cce97ddb8814" containerName="authorino"
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.814999    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d916f1-c81c-460b-8c1d-cce97ddb8814" containerName="authorino"
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.815021    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" containerName="authorino"
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.815028    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" containerName="authorino"
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.815090    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e163a1c-ee58-4d93-abf2-274bfc1c2612" containerName="authorino"
Sep 12 00:30:42.815292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.815102    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d916f1-c81c-460b-8c1d-cce97ddb8814" containerName="authorino"
Sep 12 00:30:42.817985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.817965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.821060 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.821040    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 12 00:30:42.821196 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.821038    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 12 00:30:42.821196 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.821128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rjscr\""
Sep 12 00:30:42.821350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.821333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 12 00:30:42.827345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.827321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"]
Sep 12 00:30:42.916959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.916927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh879\" (UniqueName: \"kubernetes.io/projected/9908bc37-4700-45e5-8786-724351bee1d9-kube-api-access-kh879\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.916959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.916964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9908bc37-4700-45e5-8786-724351bee1d9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.917156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.916998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.917156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.917066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.917156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.917101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:42.917156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:42.917138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018107 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh879\" (UniqueName: \"kubernetes.io/projected/9908bc37-4700-45e5-8786-724351bee1d9-kube-api-access-kh879\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9908bc37-4700-45e5-8786-724351bee1d9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.018685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.018604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.020514 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.020490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9908bc37-4700-45e5-8786-724351bee1d9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.020758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.020738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9908bc37-4700-45e5-8786-724351bee1d9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.025996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.025974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh879\" (UniqueName: \"kubernetes.io/projected/9908bc37-4700-45e5-8786-724351bee1d9-kube-api-access-kh879\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g\" (UID: \"9908bc37-4700-45e5-8786-724351bee1d9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.129225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.129141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:43.398515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.398489    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"]
Sep 12 00:30:43.400978 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:30:43.400950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9908bc37_4700_45e5_8786_724351bee1d9.slice/crio-c92d5de78d89f3fddfc9aa3b080a96d8608ffe3f8f6fc2fc6f700fafbef1af1d WatchSource:0}: Error finding container c92d5de78d89f3fddfc9aa3b080a96d8608ffe3f8f6fc2fc6f700fafbef1af1d: Status 404 returned error can't find the container with id c92d5de78d89f3fddfc9aa3b080a96d8608ffe3f8f6fc2fc6f700fafbef1af1d
Sep 12 00:30:43.402946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.402924    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:30:43.748307 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:43.748265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g" event={"ID":"9908bc37-4700-45e5-8786-724351bee1d9","Type":"ContainerStarted","Data":"c92d5de78d89f3fddfc9aa3b080a96d8608ffe3f8f6fc2fc6f700fafbef1af1d"}
Sep 12 00:30:44.755293 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:44.755206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g" event={"ID":"9908bc37-4700-45e5-8786-724351bee1d9","Type":"ContainerStarted","Data":"480736240eafcfc18477c726b1d42337e99672b9cb52b8233b226438d6d4830b"}
Sep 12 00:30:44.771401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:44.771351    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g" podStartSLOduration=1.703425396 podStartE2EDuration="2.771333899s" totalImagesPullingTime="1.067908503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:44.770600166 +0000 UTC m=+840.904590391" watchObservedRunningTime="2026-09-12 00:30:44.771333899 +0000 UTC m=+840.905324124"
Sep 12 00:30:53.129689 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:53.129643    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:53.130194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:53.129725    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:53.141902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:53.141870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:30:53.810838 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:30:53.810810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g"
Sep 12 00:31:05.225830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.225785    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"]
Sep 12 00:31:05.229225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.229208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.233627 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.233603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 12 00:31:05.251197 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.251159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"]
Sep 12 00:31:05.278965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.278932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.278965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.278961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.279182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.278979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.279182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.278998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.279182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.279100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbf5\" (UniqueName: \"kubernetes.io/projected/509f4d03-d9fc-4a53-80a9-9c396aa57acb-kube-api-access-ndbf5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.279297 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.279182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380353 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380353 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbf5\" (UniqueName: \"kubernetes.io/projected/509f4d03-d9fc-4a53-80a9-9c396aa57acb-kube-api-access-ndbf5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380855 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380855 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.380955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.380878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.382773 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.382743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509f4d03-d9fc-4a53-80a9-9c396aa57acb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.383052 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.383034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509f4d03-d9fc-4a53-80a9-9c396aa57acb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.392799 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.392761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbf5\" (UniqueName: \"kubernetes.io/projected/509f4d03-d9fc-4a53-80a9-9c396aa57acb-kube-api-access-ndbf5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5\" (UID: \"509f4d03-d9fc-4a53-80a9-9c396aa57acb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.539173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.539074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:05.811033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.810999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"]
Sep 12 00:31:05.813570 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:31:05.813531    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509f4d03_d9fc_4a53_80a9_9c396aa57acb.slice/crio-f873c2c3bcef3f81692b7b0c8a269c42ce44603bdbd977101023d6dfba5bf34e WatchSource:0}: Error finding container f873c2c3bcef3f81692b7b0c8a269c42ce44603bdbd977101023d6dfba5bf34e: Status 404 returned error can't find the container with id f873c2c3bcef3f81692b7b0c8a269c42ce44603bdbd977101023d6dfba5bf34e
Sep 12 00:31:05.865500 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:05.865458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5" event={"ID":"509f4d03-d9fc-4a53-80a9-9c396aa57acb","Type":"ContainerStarted","Data":"f873c2c3bcef3f81692b7b0c8a269c42ce44603bdbd977101023d6dfba5bf34e"}
Sep 12 00:31:06.873209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:06.873169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5" event={"ID":"509f4d03-d9fc-4a53-80a9-9c396aa57acb","Type":"ContainerStarted","Data":"fd25bb008f48fda1d2fa17dfd81c34599898cb258fd6af04235c696c0ce7f0f4"}
Sep 12 00:31:06.890965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:06.890906    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5" podStartSLOduration=1.689337292 podStartE2EDuration="1.890888017s" totalImagesPullingTime="201.550725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:31:06.889425185 +0000 UTC m=+863.023415413" watchObservedRunningTime="2026-09-12 00:31:06.890888017 +0000 UTC m=+863.024878257"
Sep 12 00:31:12.904644 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.904612    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"]
Sep 12 00:31:12.907956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.907940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.910983 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.910960    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 12 00:31:12.920425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.920400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"]
Sep 12 00:31:12.950627 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9845-bb62-471f-a1b3-8924846546d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.950627 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckksx\" (UniqueName: \"kubernetes.io/projected/ae7b9845-bb62-471f-a1b3-8924846546d0-kube-api-access-ckksx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.950863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.950863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.950863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:12.950863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:12.950828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051333 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9845-bb62-471f-a1b3-8924846546d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051333 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckksx\" (UniqueName: \"kubernetes.io/projected/ae7b9845-bb62-471f-a1b3-8924846546d0-kube-api-access-ckksx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.051939 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.051916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.053695 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.053672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae7b9845-bb62-471f-a1b3-8924846546d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.053876 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.053862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9845-bb62-471f-a1b3-8924846546d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.058440 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.058416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckksx\" (UniqueName: \"kubernetes.io/projected/ae7b9845-bb62-471f-a1b3-8924846546d0-kube-api-access-ckksx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8hqxq\" (UID: \"ae7b9845-bb62-471f-a1b3-8924846546d0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.219101 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.219072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:13.486964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.486941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"]
Sep 12 00:31:13.488560 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:31:13.488529    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7b9845_bb62_471f_a1b3_8924846546d0.slice/crio-93f1bd8fe1427b4073d3044c555bf33cd0386f759586ec8c7c1cff1edd550208 WatchSource:0}: Error finding container 93f1bd8fe1427b4073d3044c555bf33cd0386f759586ec8c7c1cff1edd550208: Status 404 returned error can't find the container with id 93f1bd8fe1427b4073d3044c555bf33cd0386f759586ec8c7c1cff1edd550208
Sep 12 00:31:13.910540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.910498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq" event={"ID":"ae7b9845-bb62-471f-a1b3-8924846546d0","Type":"ContainerStarted","Data":"05c9cf6d08c7be36142d7c22b265caacca4d7bf67c4761f5da2dd94809ddc1ec"}
Sep 12 00:31:13.910936 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.910561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq" event={"ID":"ae7b9845-bb62-471f-a1b3-8924846546d0","Type":"ContainerStarted","Data":"93f1bd8fe1427b4073d3044c555bf33cd0386f759586ec8c7c1cff1edd550208"}
Sep 12 00:31:13.930436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:13.930380    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq" podStartSLOduration=1.7330436059999998 podStartE2EDuration="1.930362777s" totalImagesPullingTime="197.319171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:31:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:31:13.927708269 +0000 UTC m=+870.061698495" watchObservedRunningTime="2026-09-12 00:31:13.930362777 +0000 UTC m=+870.064353001"
Sep 12 00:31:15.540001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:15.539963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:15.540409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:15.540027    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:15.552782 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:15.552740    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:15.932974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:15.932951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5"
Sep 12 00:31:23.219202 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:23.219162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:23.219611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:23.219320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:23.232157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:23.232137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:31:23.973620 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:31:23.973587    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8hqxq"
Sep 12 00:32:55.640457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.640421    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-595995c945-f6lzz"]
Sep 12 00:32:55.644285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.644251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.651331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.651299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595995c945-f6lzz"]
Sep 12 00:32:55.734004 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.733970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnr5k\" (UniqueName: \"kubernetes.io/projected/e94ed612-a352-4b76-9a81-a7ba95a00792-kube-api-access-gnr5k\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.734213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.734081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e94ed612-a352-4b76-9a81-a7ba95a00792-tls-cert\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.834908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.834865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnr5k\" (UniqueName: \"kubernetes.io/projected/e94ed612-a352-4b76-9a81-a7ba95a00792-kube-api-access-gnr5k\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.835100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.834965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e94ed612-a352-4b76-9a81-a7ba95a00792-tls-cert\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.837404 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.837384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e94ed612-a352-4b76-9a81-a7ba95a00792-tls-cert\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.842344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.842319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnr5k\" (UniqueName: \"kubernetes.io/projected/e94ed612-a352-4b76-9a81-a7ba95a00792-kube-api-access-gnr5k\") pod \"authorino-595995c945-f6lzz\" (UID: \"e94ed612-a352-4b76-9a81-a7ba95a00792\") " pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:55.956949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:55.956913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595995c945-f6lzz"
Sep 12 00:32:56.226095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:56.226064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595995c945-f6lzz"]
Sep 12 00:32:56.227785 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:32:56.227738    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode94ed612_a352_4b76_9a81_a7ba95a00792.slice/crio-1e81fcc18daac3022b1d7c3fbdbf69dcb78cd60886d894487f69f0be63cf845a WatchSource:0}: Error finding container 1e81fcc18daac3022b1d7c3fbdbf69dcb78cd60886d894487f69f0be63cf845a: Status 404 returned error can't find the container with id 1e81fcc18daac3022b1d7c3fbdbf69dcb78cd60886d894487f69f0be63cf845a
Sep 12 00:32:56.426589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:56.426547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595995c945-f6lzz" event={"ID":"e94ed612-a352-4b76-9a81-a7ba95a00792","Type":"ContainerStarted","Data":"1e81fcc18daac3022b1d7c3fbdbf69dcb78cd60886d894487f69f0be63cf845a"}
Sep 12 00:32:57.438591 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.438548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595995c945-f6lzz" event={"ID":"e94ed612-a352-4b76-9a81-a7ba95a00792","Type":"ContainerStarted","Data":"211e496ead456e19ad00f3ecadf8eb1c04c87635cb3ddc4d2ed3d4ca36340446"}
Sep 12 00:32:57.464559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.464500    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-595995c945-f6lzz" podStartSLOduration=1.692902915 podStartE2EDuration="2.464481516s" totalImagesPullingTime="771.578601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:32:57.463592301 +0000 UTC m=+973.597582530" watchObservedRunningTime="2026-09-12 00:32:57.464481516 +0000 UTC m=+973.598471776"
Sep 12 00:32:57.494132 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.494094    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:32:57.494378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.494315    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" podUID="d0dd5cc6-700e-4b35-95da-e7297592063a" containerName="authorino" containerID="cri-o://3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c" gracePeriod=30
Sep 12 00:32:57.750672 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.750646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:32:57.852133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.852095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw2bb\" (UniqueName: \"kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb\") pod \"d0dd5cc6-700e-4b35-95da-e7297592063a\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") "
Sep 12 00:32:57.852336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.852240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert\") pod \"d0dd5cc6-700e-4b35-95da-e7297592063a\" (UID: \"d0dd5cc6-700e-4b35-95da-e7297592063a\") "
Sep 12 00:32:57.854218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.854176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb" (OuterVolumeSpecName: "kube-api-access-xw2bb") pod "d0dd5cc6-700e-4b35-95da-e7297592063a" (UID: "d0dd5cc6-700e-4b35-95da-e7297592063a"). InnerVolumeSpecName "kube-api-access-xw2bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:32:57.862304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.862277    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d0dd5cc6-700e-4b35-95da-e7297592063a" (UID: "d0dd5cc6-700e-4b35-95da-e7297592063a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:32:57.953395 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.953356    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0dd5cc6-700e-4b35-95da-e7297592063a-tls-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:32:57.953395 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:57.953393    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw2bb\" (UniqueName: \"kubernetes.io/projected/d0dd5cc6-700e-4b35-95da-e7297592063a-kube-api-access-xw2bb\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:32:58.445569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.445532    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0dd5cc6-700e-4b35-95da-e7297592063a" containerID="3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c" exitCode=0
Sep 12 00:32:58.446046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.445585    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c9b9788-sqnnc"
Sep 12 00:32:58.446046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.445625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" event={"ID":"d0dd5cc6-700e-4b35-95da-e7297592063a","Type":"ContainerDied","Data":"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"}
Sep 12 00:32:58.446046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.445675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c9b9788-sqnnc" event={"ID":"d0dd5cc6-700e-4b35-95da-e7297592063a","Type":"ContainerDied","Data":"0c73a27125296e7b1a4b70b809e2da616c42e0d61ef6cb5150aef8444703f8cf"}
Sep 12 00:32:58.446046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.445698    2580 scope.go:117] "RemoveContainer" containerID="3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"
Sep 12 00:32:58.454677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.454654    2580 scope.go:117] "RemoveContainer" containerID="3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"
Sep 12 00:32:58.454982 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:32:58.454961    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c\": container with ID starting with 3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c not found: ID does not exist" containerID="3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"
Sep 12 00:32:58.455067 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.454995    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c"} err="failed to get container status \"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c\": rpc error: code = NotFound desc = could not find container \"3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c\": container with ID starting with 3e6f3ab56c17a13e8df98c12ec86206f1e73c251dbadb966917de7a171d7c18c not found: ID does not exist"
Sep 12 00:32:58.460893 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.460866    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:32:58.466430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:32:58.466407    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b6c9b9788-sqnnc"]
Sep 12 00:33:00.373212 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:00.373175    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0dd5cc6-700e-4b35-95da-e7297592063a" path="/var/lib/kubelet/pods/d0dd5cc6-700e-4b35-95da-e7297592063a/volumes"
Sep 12 00:33:15.543105 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.543010    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:33:15.543638 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.543541    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0dd5cc6-700e-4b35-95da-e7297592063a" containerName="authorino"
Sep 12 00:33:15.543638 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.543562    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0dd5cc6-700e-4b35-95da-e7297592063a" containerName="authorino"
Sep 12 00:33:15.543753 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.543671    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0dd5cc6-700e-4b35-95da-e7297592063a" containerName="authorino"
Sep 12 00:33:15.546795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.546746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.549128 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.549093    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-3c466c-openshift-default-dockercfg-fp79l\""
Sep 12 00:33:15.559646 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.559620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:33:15.613020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.612976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th7mc\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.613492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.613332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714466 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714653 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.714964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.714723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th7mc\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.715113 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.715078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.715169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.715111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.715318 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.715294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.715444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.715346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.715934 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.715912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.717310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.717291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.717654 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.717637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.722397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.722373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th7mc\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.722591 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.722568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token\") pod \"e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") " pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:15.857422 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:15.857330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:16.137997 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.137972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:33:16.139888 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:16.139856    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9271239b_19a7_4dc2_8134_2eecc27bc446.slice/crio-d8a5e08e82096d4c9f0cc386c4b6c25857f8a875bfb10f1fb24b82054dee6ed6 WatchSource:0}: Error finding container d8a5e08e82096d4c9f0cc386c4b6c25857f8a875bfb10f1fb24b82054dee6ed6: Status 404 returned error can't find the container with id d8a5e08e82096d4c9f0cc386c4b6c25857f8a875bfb10f1fb24b82054dee6ed6
Sep 12 00:33:16.142345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.142311    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:16.142419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.142393    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:16.142464 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.142433    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:16.542961 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.542925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerStarted","Data":"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"}
Sep 12 00:33:16.543136 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.542977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerStarted","Data":"d8a5e08e82096d4c9f0cc386c4b6c25857f8a875bfb10f1fb24b82054dee6ed6"}
Sep 12 00:33:16.562992 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.562947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podStartSLOduration=1.562929019 podStartE2EDuration="1.562929019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:16.561584606 +0000 UTC m=+992.695574824" watchObservedRunningTime="2026-09-12 00:33:16.562929019 +0000 UTC m=+992.696919244"
Sep 12 00:33:16.857610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:16.857524    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:17.863815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:17.863780    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:18.552075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:18.552037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:18.552896 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:18.552875    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:20.607263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.607224    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7894-9be82f"]
Sep 12 00:33:20.610569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.610551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:20.612614 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.612594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qwc4m\""
Sep 12 00:33:20.616960 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.616935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7894-9be82f"]
Sep 12 00:33:20.763562 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.763519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcnc\" (UniqueName: \"kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc\") pod \"test-curl-7894-9be82f\" (UID: \"03a14419-8c6f-4e7f-85f8-615ddfde6414\") " pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:20.864994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.864909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcnc\" (UniqueName: \"kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc\") pod \"test-curl-7894-9be82f\" (UID: \"03a14419-8c6f-4e7f-85f8-615ddfde6414\") " pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:20.872314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.872281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcnc\" (UniqueName: \"kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc\") pod \"test-curl-7894-9be82f\" (UID: \"03a14419-8c6f-4e7f-85f8-615ddfde6414\") " pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:20.922345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:20.922307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:21.196109 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:21.196080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7894-9be82f"]
Sep 12 00:33:21.197722 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:21.197698    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03a14419_8c6f_4e7f_85f8_615ddfde6414.slice/crio-8d2d9539f4bc5c162378bd2ff25fbe5824b81308ba39510f172e3c8b14627dd0 WatchSource:0}: Error finding container 8d2d9539f4bc5c162378bd2ff25fbe5824b81308ba39510f172e3c8b14627dd0: Status 404 returned error can't find the container with id 8d2d9539f4bc5c162378bd2ff25fbe5824b81308ba39510f172e3c8b14627dd0
Sep 12 00:33:21.569667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:21.569580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7894-9be82f" event={"ID":"03a14419-8c6f-4e7f-85f8-615ddfde6414","Type":"ContainerStarted","Data":"8d2d9539f4bc5c162378bd2ff25fbe5824b81308ba39510f172e3c8b14627dd0"}
Sep 12 00:33:22.576806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.576750    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7894-9be82f" event={"ID":"03a14419-8c6f-4e7f-85f8-615ddfde6414","Type":"ContainerStarted","Data":"8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84"}
Sep 12 00:33:22.591492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.591444    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7894-9be82f" podStartSLOduration=2.04977002 podStartE2EDuration="2.59142596s" totalImagesPullingTime="541.65594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:22.589611716 +0000 UTC m=+998.723601941" watchObservedRunningTime="2026-09-12 00:33:22.59142596 +0000 UTC m=+998.725416201"
Sep 12 00:33:22.638710 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.638679    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:33:22.642282 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.642265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.644418 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.644396    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 12 00:33:22.644534 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.644414    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 12 00:33:22.644534 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.644441    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 00:33:22.645122 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.645100    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-139288-serving-cert\""
Sep 12 00:33:22.645219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.645111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f4scw\""
Sep 12 00:33:22.653284 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.653260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:33:22.784137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.784098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.784296 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.784165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.784296 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.784223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.784392 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.784306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhzpw\" (UniqueName: \"kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.884795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.884699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhzpw\" (UniqueName: \"kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.884795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.884736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.884795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.884787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.885077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.884834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.885077 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:22.884963    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-serving-cert: secret "maas-api-e2e-worker-w4-139288-serving-cert" not found
Sep 12 00:33:22.885077 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:22.885032    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls podName:1ddcc8f9-93b0-4222-af14-062f437c2b21 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:23.385012499 +0000 UTC m=+999.519002719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls") pod "maas-api-e2e-worker-w4-139288-588856d668-2mtnx" (UID: "1ddcc8f9-93b0-4222-af14-062f437c2b21") : secret "maas-api-e2e-worker-w4-139288-serving-cert" not found
Sep 12 00:33:22.885494 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.885468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.887161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.887141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:22.892817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:22.892797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhzpw\" (UniqueName: \"kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:23.237722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.237689    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7894-9be82f"]
Sep 12 00:33:23.242119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.242088    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7894-9be82f"]
Sep 12 00:33:23.390718 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.390666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:23.393385 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.393350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") pod \"maas-api-e2e-worker-w4-139288-588856d668-2mtnx\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:23.555373 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.555289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:23.582711 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:23.582683    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7894-9be82f" container="test-curl-7894-9be82f"
Sep 12 00:33:23.828344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:23.828317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:33:23.829956 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:23.829929    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddcc8f9_93b0_4222_af14_062f437c2b21.slice/crio-bb039291ee65e4a5f460b7af711e62aa209b4a87e98cb5747ee862bce51931ef WatchSource:0}: Error finding container bb039291ee65e4a5f460b7af711e62aa209b4a87e98cb5747ee862bce51931ef: Status 404 returned error can't find the container with id bb039291ee65e4a5f460b7af711e62aa209b4a87e98cb5747ee862bce51931ef
Sep 12 00:33:24.047006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.046969    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:33:24.047740 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.047712    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" containerName="test-curl-7894-9be82f"
Sep 12 00:33:24.047881 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.047760    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" containerName="test-curl-7894-9be82f"
Sep 12 00:33:24.047881 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.047869    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" containerName="test-curl-7894-9be82f"
Sep 12 00:33:24.052221 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.052200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.054368 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.054348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-4aaf1a-openshift-default-dockercfg-sqdww\""
Sep 12 00:33:24.063363 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.063340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:33:24.196960 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.196925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.196983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhcjj\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.197290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.197272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhcjj\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298871 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298871 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.298989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.298968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.299175 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.299158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.300827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.300805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.301742 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.301719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.306077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.306049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.306160 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.306095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhcjj\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj\") pod \"e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") " pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.364696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.364650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:24.384001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.382841    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:33:24.388517 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.388499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.391467 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.391119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1b1c80d8-openshift-default-dockercfg-chh5x\""
Sep 12 00:33:24.402652 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.402626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.500738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.500917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.500962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s87p4\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.501275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.502062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.501384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.590036 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.589996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" event={"ID":"1ddcc8f9-93b0-4222-af14-062f437c2b21","Type":"ContainerStarted","Data":"bb039291ee65e4a5f460b7af711e62aa209b4a87e98cb5747ee862bce51931ef"}
Sep 12 00:33:24.590513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.590146    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7894-9be82f" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" containerName="test-curl-7894-9be82f" containerID="cri-o://8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84" gracePeriod=2
Sep 12 00:33:24.604571 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604571 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87p4\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.604803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.604858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.605078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.605274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.605436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605580 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.605551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.605975 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.605889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.607458 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.607436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.607710 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.607687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.612214 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.612170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87p4\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.612318 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.612300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token\") pod \"e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") " pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:24.674457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.674401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:33:24.678360 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:24.678315    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda816e271_0eb7_441e_81e7_0e334d5e0a12.slice/crio-47a5b835b37ebea7165ec30cf2b4916326e0ef063bcc03fe336e5c391b7e1037 WatchSource:0}: Error finding container 47a5b835b37ebea7165ec30cf2b4916326e0ef063bcc03fe336e5c391b7e1037: Status 404 returned error can't find the container with id 47a5b835b37ebea7165ec30cf2b4916326e0ef063bcc03fe336e5c391b7e1037
Sep 12 00:33:24.681190 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.681159    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:24.681274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.681222    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:24.681274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.681253    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:24.712139 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:24.712084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:25.011928 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.011729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:33:25.015107 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:25.015076    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef2bb2f_c5c8_4507_9114_7d9d0406061d.slice/crio-cd81472f68b6c45f1e3faa7d2e2be9a8eabd9393e1add11f04b737135c652482 WatchSource:0}: Error finding container cd81472f68b6c45f1e3faa7d2e2be9a8eabd9393e1add11f04b737135c652482: Status 404 returned error can't find the container with id cd81472f68b6c45f1e3faa7d2e2be9a8eabd9393e1add11f04b737135c652482
Sep 12 00:33:25.018092 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.018053    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:25.018193 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.018141    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:25.018193 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.018182    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:25.601735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.601668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerStarted","Data":"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"}
Sep 12 00:33:25.602636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.601824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerStarted","Data":"cd81472f68b6c45f1e3faa7d2e2be9a8eabd9393e1add11f04b737135c652482"}
Sep 12 00:33:25.605495 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.605440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerStarted","Data":"d3278751c2704a53d4e54b56622399c4228271d128124e5967dacc3447647351"}
Sep 12 00:33:25.605618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.605525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerStarted","Data":"47a5b835b37ebea7165ec30cf2b4916326e0ef063bcc03fe336e5c391b7e1037"}
Sep 12 00:33:25.623825 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.623751    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podStartSLOduration=1.623731093 podStartE2EDuration="1.623731093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:25.620964447 +0000 UTC m=+1001.754954674" watchObservedRunningTime="2026-09-12 00:33:25.623731093 +0000 UTC m=+1001.757721321"
Sep 12 00:33:25.642424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.642373    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podStartSLOduration=1.642354233 podStartE2EDuration="1.642354233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:25.639133949 +0000 UTC m=+1001.773124175" watchObservedRunningTime="2026-09-12 00:33:25.642354233 +0000 UTC m=+1001.776344460"
Sep 12 00:33:25.712379 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:25.712343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:26.366070 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.366044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:26.374983 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.374961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:26.613632 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.613524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" event={"ID":"1ddcc8f9-93b0-4222-af14-062f437c2b21","Type":"ContainerStarted","Data":"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"}
Sep 12 00:33:26.614155 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.614130    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:26.614932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.614912    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:26.635985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.635931    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" podStartSLOduration=2.105192853 podStartE2EDuration="4.635913084s" totalImagesPullingTime="2.530720231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:26.633047765 +0000 UTC m=+1002.767037991" watchObservedRunningTime="2026-09-12 00:33:26.635913084 +0000 UTC m=+1002.769903365"
Sep 12 00:33:26.746483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.746452    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:26.835515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.835484    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:26.945459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.945429    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftcnc\" (UniqueName: \"kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc\") pod \"03a14419-8c6f-4e7f-85f8-615ddfde6414\" (UID: \"03a14419-8c6f-4e7f-85f8-615ddfde6414\") "
Sep 12 00:33:26.947617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:26.947595    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc" (OuterVolumeSpecName: "kube-api-access-ftcnc") pod "03a14419-8c6f-4e7f-85f8-615ddfde6414" (UID: "03a14419-8c6f-4e7f-85f8-615ddfde6414"). InnerVolumeSpecName "kube-api-access-ftcnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:33:27.046184 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.046147    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftcnc\" (UniqueName: \"kubernetes.io/projected/03a14419-8c6f-4e7f-85f8-615ddfde6414-kube-api-access-ftcnc\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:33:27.620600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.620566    2580 generic.go:356] "Generic (PLEG): container finished" podID="03a14419-8c6f-4e7f-85f8-615ddfde6414" containerID="8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84" exitCode=137
Sep 12 00:33:27.621037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.620606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:27.621037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.620660    2580 scope.go:117] "RemoveContainer" containerID="8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84"
Sep 12 00:33:27.621171 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.621153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:27.621492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.621453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:27.622449 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.622426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:27.630635 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.630607    2580 scope.go:117] "RemoveContainer" containerID="8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84"
Sep 12 00:33:27.630981 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:27.630954    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84\": container with ID starting with 8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84 not found: ID does not exist" containerID="8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84"
Sep 12 00:33:27.631072 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:27.630990    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84"} err="failed to get container status \"8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84\": rpc error: code = NotFound desc = could not find container \"8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84\": container with ID starting with 8672710f2d77853ad31ea4d058bc37633761ca7b025f18662398458a77e1ab84 not found: ID does not exist"
Sep 12 00:33:27.646813 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:27.646783    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7894-9be82f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" pod="opendatahub/test-curl-7894-9be82f"
Sep 12 00:33:28.374377 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:28.374344    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a14419-8c6f-4e7f-85f8-615ddfde6414" path="/var/lib/kubelet/pods/03a14419-8c6f-4e7f-85f8-615ddfde6414/volumes"
Sep 12 00:33:33.178419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.178372    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:33:33.183486 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.183466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.185651 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.185633    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1b1c80d8\""
Sep 12 00:33:33.194440 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.194410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.194555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.194456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pddk\" (UniqueName: \"kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.195908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.195886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:33:33.295361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.295326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.295504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.295373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pddk\" (UniqueName: \"kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.296159 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.296139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.303241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.303215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pddk\" (UniqueName: \"kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk\") pod \"payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") " pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.494971 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.494877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:33.641647 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.641552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:33:33.792165 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:33.792135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:33:33.793991 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:33:33.793958    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afede32_a94f_4e91_9be9_ce86f1b8c7d9.slice/crio-e754a6403ac7c3b4e6f9d7e76e13003d0d3a2f981157004e83164d0a610f9f47 WatchSource:0}: Error finding container e754a6403ac7c3b4e6f9d7e76e13003d0d3a2f981157004e83164d0a610f9f47: Status 404 returned error can't find the container with id e754a6403ac7c3b4e6f9d7e76e13003d0d3a2f981157004e83164d0a610f9f47
Sep 12 00:33:34.669607 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:34.669556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" event={"ID":"3afede32-a94f-4e91-9be9-ce86f1b8c7d9","Type":"ContainerStarted","Data":"e754a6403ac7c3b4e6f9d7e76e13003d0d3a2f981157004e83164d0a610f9f47"}
Sep 12 00:33:35.678505 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:35.678462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" event={"ID":"3afede32-a94f-4e91-9be9-ce86f1b8c7d9","Type":"ContainerStarted","Data":"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"}
Sep 12 00:33:35.678997 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:35.678529    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:35.697532 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:35.697451    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" podStartSLOduration=1.2873146819999999 podStartE2EDuration="2.697429113s" totalImagesPullingTime="1.410114431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:35.693579595 +0000 UTC m=+1011.827569821" watchObservedRunningTime="2026-09-12 00:33:35.697429113 +0000 UTC m=+1011.831419340"
Sep 12 00:33:39.086499 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.085233    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:33:39.088261 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.088236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.091161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.091136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-39d42b-openshift-default-dockercfg-n5bj5\""
Sep 12 00:33:39.099548 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.099517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.161822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.161881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.161928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.161953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.161990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.162018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.162072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.162116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.162791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.162141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48pqv\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.263845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.263468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.263845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.263533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.263845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.263578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.263845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.263623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.263845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.263698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.264259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.264269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.264325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48pqv\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.264418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.264447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.265230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.265470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.265739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.265818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.267033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.268792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.267609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.275157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.275105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48pqv\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.277121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.277098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token\") pod \"e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") " pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.400843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.400813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:39.711733 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.711697    2580 generic.go:356] "Generic (PLEG): container finished" podID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerID="d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988" exitCode=0
Sep 12 00:33:39.711733 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.711738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerDied","Data":"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"}
Sep 12 00:33:39.712206 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.712190    2580 scope.go:117] "RemoveContainer" containerID="d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"
Sep 12 00:33:39.713306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.713270    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:39.713433 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.713354    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:39.713433 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.713404    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:39.732358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.732312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:33:39.778226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.778171    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:39.778401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.778270    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:39.778401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:39.778315    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:40.721628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.721583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerStarted","Data":"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"}
Sep 12 00:33:40.725106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.725066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerStarted","Data":"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"}
Sep 12 00:33:40.725267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.725128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerStarted","Data":"5e979a460bed23d13168bd21f39784268838987043207fa274f194461f1991fc"}
Sep 12 00:33:40.760734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.760653    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podStartSLOduration=1.7606165379999998 podStartE2EDuration="1.760616538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:40.760020765 +0000 UTC m=+1016.894010986" watchObservedRunningTime="2026-09-12 00:33:40.760616538 +0000 UTC m=+1016.894606764"
Sep 12 00:33:40.857735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.857691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:40.858121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.858089    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:40.858239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:40.858173    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:41.402869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.402835    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:41.406702 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.406676    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:41.734542 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.734515    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:41.746315 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.746277    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:33:41.857989 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.857946    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:41.858193 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:41.858030    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:42.857997 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:42.857950    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:42.858509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:42.858023    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:43.858424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:43.858375    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:43.858921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:43.858435    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:44.858556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:44.858504    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:44.859115 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:44.858579    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:45.858239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:45.858199    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:45.858430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:45.858304    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 12 00:33:46.696625 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:46.696593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:33:46.863162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:46.863133    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:46.863432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:46.863406    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:46.864173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:46.864152    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:33:46.912251 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:33:46.912120    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef2bb2f_c5c8_4507_9114_7d9d0406061d.slice/crio-conmon-5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef2bb2f_c5c8_4507_9114_7d9d0406061d.slice/crio-5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62.scope\": RecentStats: unable to find data in memory cache]"
Sep 12 00:33:47.779066 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.779034    2580 generic.go:356] "Generic (PLEG): container finished" podID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerID="5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62" exitCode=0
Sep 12 00:33:47.779451 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.779108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerDied","Data":"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"}
Sep 12 00:33:47.779573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.779558    2580 scope.go:117] "RemoveContainer" containerID="5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"
Sep 12 00:33:47.780696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.780658    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:47.780857 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.780751    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:47.780857 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:47.780818    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:48.786735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.786698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerStarted","Data":"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"}
Sep 12 00:33:48.788419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.788396    2580 generic.go:356] "Generic (PLEG): container finished" podID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerID="d3278751c2704a53d4e54b56622399c4228271d128124e5967dacc3447647351" exitCode=0
Sep 12 00:33:48.788520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.788469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerDied","Data":"d3278751c2704a53d4e54b56622399c4228271d128124e5967dacc3447647351"}
Sep 12 00:33:48.788852 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.788839    2580 scope.go:117] "RemoveContainer" containerID="d3278751c2704a53d4e54b56622399c4228271d128124e5967dacc3447647351"
Sep 12 00:33:48.789864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.789834    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:48.789968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.789917    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:48.790040 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:48.789969    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:33:49.712828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:49.712792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:49.713019 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:49.713001    2580 patch_prober.go:28] interesting pod/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 12 00:33:49.713067 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:49.713052    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 12 00:33:49.795603 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:49.795568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerStarted","Data":"cde52180ea277012535336143276c7c5230aa348ff185a865369fc124ab18290"}
Sep 12 00:33:50.365302 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:50.365197    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:50.365522 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:50.365474    2580 patch_prober.go:28] interesting pod/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 12 00:33:50.365596 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:50.365547    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 12 00:33:50.712914 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:50.712876    2580 patch_prober.go:28] interesting pod/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 12 00:33:50.713094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:50.712937    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 12 00:33:51.365512 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:51.365457    2580 patch_prober.go:28] interesting pod/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 12 00:33:51.366040 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:51.365552    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 12 00:33:51.713316 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:51.713272    2580 patch_prober.go:28] interesting pod/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 12 00:33:51.713496 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:51.713340    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 12 00:33:52.365784 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:52.365719    2580 patch_prober.go:28] interesting pod/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 12 00:33:52.366427 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:52.366391    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 12 00:33:52.712616 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:52.712573    2580 patch_prober.go:28] interesting pod/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 12 00:33:52.712827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:52.712632    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 12 00:33:53.366366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:53.366321    2580 patch_prober.go:28] interesting pod/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 12 00:33:53.366980 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:53.366947    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 12 00:33:53.712759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:53.712720    2580 patch_prober.go:28] interesting pod/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 12 00:33:53.712967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:53.712802    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 12 00:33:54.365959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:54.365915    2580 patch_prober.go:28] interesting pod/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 12 00:33:54.366169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:54.365990    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 12 00:33:54.722959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:54.722922    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:54.724822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:54.724798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:54.732130 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:54.732106    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:33:55.369060 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:55.369026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:55.369331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:55.369309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:33:55.370120 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:33:55.370104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:34:01.878719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.878682    2580 generic.go:356] "Generic (PLEG): container finished" podID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerID="1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20" exitCode=0
Sep 12 00:34:01.879154 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.878752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerDied","Data":"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"}
Sep 12 00:34:01.879197 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.879165    2580 scope.go:117] "RemoveContainer" containerID="1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"
Sep 12 00:34:01.880376 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.880348    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:01.880444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.880419    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:01.880483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:01.880447    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:02.888859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:02.888815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerStarted","Data":"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"}
Sep 12 00:34:03.401512 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:03.401465    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:34:03.401697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:03.401681    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:03.401744 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:03.401725    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:04.401791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:04.401727    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:04.402180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:04.401808    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:05.402351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:05.402316    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:05.402747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:05.402377    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:06.402420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:06.402380    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:06.402855 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:06.402439    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:07.401887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:07.401836    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:07.402075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:07.401931    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:08.401506 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:08.401452    2580 patch_prober.go:28] interesting pod/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 12 00:34:08.401997 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:08.401524    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 12 00:34:09.406238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:09.406207    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:34:09.406667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:09.406569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:34:09.407347 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:09.407326    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:34:19.034995 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.034963    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:34:19.035585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.035184    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" podUID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" containerName="payload-processing" containerID="cri-o://c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632" gracePeriod=30
Sep 12 00:34:19.284287 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.284255    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:34:19.343600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.343513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:34:19.343860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.343829    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy" containerID="cri-o://78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6" gracePeriod=30
Sep 12 00:34:19.396928 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.396899    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pddk\" (UniqueName: \"kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk\") pod \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") "
Sep 12 00:34:19.397079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.396951    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume\") pod \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\" (UID: \"3afede32-a94f-4e91-9be9-ce86f1b8c7d9\") "
Sep 12 00:34:19.397465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.397432    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3afede32-a94f-4e91-9be9-ce86f1b8c7d9" (UID: "3afede32-a94f-4e91-9be9-ce86f1b8c7d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:19.399215 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.399189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk" (OuterVolumeSpecName: "kube-api-access-8pddk") pod "3afede32-a94f-4e91-9be9-ce86f1b8c7d9" (UID: "3afede32-a94f-4e91-9be9-ce86f1b8c7d9"). InnerVolumeSpecName "kube-api-access-8pddk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:19.498906 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.498868    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pddk\" (UniqueName: \"kubernetes.io/projected/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-kube-api-access-8pddk\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.499151 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:19.499133    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3afede32-a94f-4e91-9be9-ce86f1b8c7d9-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:20.008736 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.008700    2580 generic.go:356] "Generic (PLEG): container finished" podID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" containerID="c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632" exitCode=0
Sep 12 00:34:20.008967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.008741    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"
Sep 12 00:34:20.008967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.008796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" event={"ID":"3afede32-a94f-4e91-9be9-ce86f1b8c7d9","Type":"ContainerDied","Data":"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"}
Sep 12 00:34:20.008967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.008855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg" event={"ID":"3afede32-a94f-4e91-9be9-ce86f1b8c7d9","Type":"ContainerDied","Data":"e754a6403ac7c3b4e6f9d7e76e13003d0d3a2f981157004e83164d0a610f9f47"}
Sep 12 00:34:20.008967 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.008883    2580 scope.go:117] "RemoveContainer" containerID="c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"
Sep 12 00:34:20.018838 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.018812    2580 scope.go:117] "RemoveContainer" containerID="c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"
Sep 12 00:34:20.019132 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:20.019111    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632\": container with ID starting with c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632 not found: ID does not exist" containerID="c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"
Sep 12 00:34:20.019187 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.019142    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632"} err="failed to get container status \"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632\": rpc error: code = NotFound desc = could not find container \"c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632\": container with ID starting with c2998ddc5791af6a8633e7732bd938ef280039a986a93b7b259b2226ebbd2632 not found: ID does not exist"
Sep 12 00:34:20.032369 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.032341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:34:20.036518 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.036492    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1b1c80d8-5ddcb5c66f-77xmg"]
Sep 12 00:34:20.374785 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.374690    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" path="/var/lib/kubelet/pods/3afede32-a94f-4e91-9be9-ce86f1b8c7d9/volumes"
Sep 12 00:34:20.424597 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.424564    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:34:20.425130 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.425112    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" containerName="payload-processing"
Sep 12 00:34:20.425181 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.425132    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" containerName="payload-processing"
Sep 12 00:34:20.425219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.425210    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3afede32-a94f-4e91-9be9-ce86f1b8c7d9" containerName="payload-processing"
Sep 12 00:34:20.429423 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.429405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.432246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.432223    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-e6d0c6f2-gw-openshift-default-dockercfg-8n4bf\""
Sep 12 00:34:20.443843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.443812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:34:20.609294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609294 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609545 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609545 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbpwq\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609545 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.609677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.609658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.710785 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbpwq\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.710959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711441 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711441 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711553 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711553 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.711675 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.711655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.713181 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.713162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.713491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.713468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.717962 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.717943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.718189 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.718169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbpwq\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq\") pod \"e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") " pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:20.739951 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:20.739928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:21.015837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:21.015808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:34:21.017159 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:34:21.017133    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0536a0ce_03c6_4b39_9b6d_e088d310de1c.slice/crio-d32b26541084d100f15b66cf9828e5b55c5ab16976903e992502bd5a49504219 WatchSource:0}: Error finding container d32b26541084d100f15b66cf9828e5b55c5ab16976903e992502bd5a49504219: Status 404 returned error can't find the container with id d32b26541084d100f15b66cf9828e5b55c5ab16976903e992502bd5a49504219
Sep 12 00:34:21.019355 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:21.019325    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:21.019436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:21.019387    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:21.019436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:21.019411    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:22.027247 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:22.027206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerStarted","Data":"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"}
Sep 12 00:34:22.027759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:22.027272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerStarted","Data":"d32b26541084d100f15b66cf9828e5b55c5ab16976903e992502bd5a49504219"}
Sep 12 00:34:22.046819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:22.046746    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podStartSLOduration=2.046728474 podStartE2EDuration="2.046728474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:22.045121632 +0000 UTC m=+1058.179111861" watchObservedRunningTime="2026-09-12 00:34:22.046728474 +0000 UTC m=+1058.180718698"
Sep 12 00:34:22.740846 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:22.740804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:22.747097 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:22.747069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:23.033141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:23.033052    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:23.034021 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:23.034000    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:24.366103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.366066    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:24.370929 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.370890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.373267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.373245    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e6d0c6f2-serving-cert\""
Sep 12 00:34:24.380829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.380801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:24.448578 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.448531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.448721 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.448600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.448721 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.448681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.448850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.448814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65w85\" (UniqueName: \"kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.549603 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.549557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.549803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.549610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.549803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.549635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.549803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.549682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65w85\" (UniqueName: \"kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.549803 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:24.549784    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-serving-cert: secret "maas-api-e2e-migrate-e6d0c6f2-serving-cert" not found
Sep 12 00:34:24.550040 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:24.549864    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls podName:c0602472-6bb2-4f33-8e94-f7613f8ae1d3 nodeName:}" failed. No retries permitted until 2026-09-12 00:34:25.049842284 +0000 UTC m=+1061.183832490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls") pod "maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" (UID: "c0602472-6bb2-4f33-8e94-f7613f8ae1d3") : secret "maas-api-e2e-migrate-e6d0c6f2-serving-cert" not found
Sep 12 00:34:24.550329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.550310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.552033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.552009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.558101 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.558078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65w85\" (UniqueName: \"kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:24.702633 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.702602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:34:24.752336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752301    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s87p4\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752434    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752622    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket\") pod \"def2bb2f-c5c8-4507-9114-7d9d0406061d\" (UID: \"def2bb2f-c5c8-4507-9114-7d9d0406061d\") "
Sep 12 00:34:24.752843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:34:24.753014 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:34:24.753014 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.752998    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.753124 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.753032    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:24.753181 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.753153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data" (OuterVolumeSpecName: "istio-data") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:34:24.753239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.753212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:34:24.754590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.754564    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:34:24.754590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.754581    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4" (OuterVolumeSpecName: "kube-api-access-s87p4") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "kube-api-access-s87p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:24.754999 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.754977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:34:24.755072 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.754980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token" (OuterVolumeSpecName: "istio-token") pod "def2bb2f-c5c8-4507-9114-7d9d0406061d" (UID: "def2bb2f-c5c8-4507-9114-7d9d0406061d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:24.853829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853760    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.853829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853824    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.853829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853835    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.854083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853848    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/def2bb2f-c5c8-4507-9114-7d9d0406061d-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.854083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853860    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.854083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853872    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s87p4\" (UniqueName: \"kubernetes.io/projected/def2bb2f-c5c8-4507-9114-7d9d0406061d-kube-api-access-s87p4\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.854083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853886    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:24.854083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:24.853898    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/def2bb2f-c5c8-4507-9114-7d9d0406061d-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:25.047272 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.047182    2580 generic.go:356] "Generic (PLEG): container finished" podID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerID="78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6" exitCode=0
Sep 12 00:34:25.047272 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.047229    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"
Sep 12 00:34:25.047454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.047264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerDied","Data":"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"}
Sep 12 00:34:25.047454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.047318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2" event={"ID":"def2bb2f-c5c8-4507-9114-7d9d0406061d","Type":"ContainerDied","Data":"cd81472f68b6c45f1e3faa7d2e2be9a8eabd9393e1add11f04b737135c652482"}
Sep 12 00:34:25.047454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.047342    2580 scope.go:117] "RemoveContainer" containerID="78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"
Sep 12 00:34:25.051529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.051502    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:25.051965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.051943    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:25.051965 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.051963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:25.052121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.051984    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:25.052121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.051993    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:25.052121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.052084    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:25.055551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.055518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:25.056520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.056495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.058140 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.058123    2580 scope.go:117] "RemoveContainer" containerID="5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"
Sep 12 00:34:25.058239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.058222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:25.058905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.058885    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e6d0c6f2-dockercfg-7rswm\""
Sep 12 00:34:25.058980 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.058885    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e6d0c6f2\""
Sep 12 00:34:25.065926 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.065899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:25.069987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.069967    2580 scope.go:117] "RemoveContainer" containerID="78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"
Sep 12 00:34:25.070331 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:25.070310    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6\": container with ID starting with 78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6 not found: ID does not exist" containerID="78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"
Sep 12 00:34:25.070407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.070342    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6"} err="failed to get container status \"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6\": rpc error: code = NotFound desc = could not find container \"78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6\": container with ID starting with 78a13461b93dcebf90638d5ecd19686875eff99edcf984978824df59c794daf6 not found: ID does not exist"
Sep 12 00:34:25.070407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.070359    2580 scope.go:117] "RemoveContainer" containerID="5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"
Sep 12 00:34:25.070748 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:25.070715    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62\": container with ID starting with 5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62 not found: ID does not exist" containerID="5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"
Sep 12 00:34:25.070926 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.070887    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62"} err="failed to get container status \"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62\": rpc error: code = NotFound desc = could not find container \"5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62\": container with ID starting with 5c28445389a89949bff080606c60719db8f6d8889c1b7285a77d0e08786c9e62 not found: ID does not exist"
Sep 12 00:34:25.093444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.093414    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:34:25.097217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.097156    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1b1c80d8-openshift-default-d6c5c7f6b-jh7z2"]
Sep 12 00:34:25.157039 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.156998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.157216 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.157129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf4g9\" (UniqueName: \"kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.258731 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.258691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.258955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.258751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf4g9\" (UniqueName: \"kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.259480 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.259453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.266508 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.266487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf4g9\" (UniqueName: \"kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9\") pod \"payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") " pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.284835 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.284811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:25.368538 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.368508    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:25.561623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.561534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:25.643139 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:25.643111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:25.644447 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:34:25.644419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75e8bef_52ee_41f2_819a_3819ac314813.slice/crio-070a693441352e523e83f954dc9f324157de1fb939e84dd61141a9aedc1df6c5 WatchSource:0}: Error finding container 070a693441352e523e83f954dc9f324157de1fb939e84dd61141a9aedc1df6c5: Status 404 returned error can't find the container with id 070a693441352e523e83f954dc9f324157de1fb939e84dd61141a9aedc1df6c5
Sep 12 00:34:26.054065 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.054029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" event={"ID":"b75e8bef-52ee-41f2-819a-3819ac314813","Type":"ContainerStarted","Data":"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"}
Sep 12 00:34:26.054262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.054091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" event={"ID":"b75e8bef-52ee-41f2-819a-3819ac314813","Type":"ContainerStarted","Data":"070a693441352e523e83f954dc9f324157de1fb939e84dd61141a9aedc1df6c5"}
Sep 12 00:34:26.054262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.054124    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:26.055692 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.055670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" event={"ID":"c0602472-6bb2-4f33-8e94-f7613f8ae1d3","Type":"ContainerStarted","Data":"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"}
Sep 12 00:34:26.055844 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.055708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" event={"ID":"c0602472-6bb2-4f33-8e94-f7613f8ae1d3","Type":"ContainerStarted","Data":"584c1aa831f66b8107c7a0f6ff0a26de52f2a954c54cba55dfb188b1fab5fb21"}
Sep 12 00:34:26.055844 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.055796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:26.069601 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.069554    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" podStartSLOduration=1.069538656 podStartE2EDuration="1.069538656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:26.068164432 +0000 UTC m=+1062.202154656" watchObservedRunningTime="2026-09-12 00:34:26.069538656 +0000 UTC m=+1062.203528880"
Sep 12 00:34:26.084556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.084506    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" podStartSLOduration=1.7015814580000002 podStartE2EDuration="2.084491836s" totalImagesPullingTime="382.910378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:26.08279617 +0000 UTC m=+1062.216786394" watchObservedRunningTime="2026-09-12 00:34:26.084491836 +0000 UTC m=+1062.218482060"
Sep 12 00:34:26.375293 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:26.375216    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" path="/var/lib/kubelet/pods/def2bb2f-c5c8-4507-9114-7d9d0406061d/volumes"
Sep 12 00:34:32.067519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:32.067485    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:37.064532 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:37.064448    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:44.177180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.177140    2580 generic.go:356] "Generic (PLEG): container finished" podID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerID="4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043" exitCode=0
Sep 12 00:34:44.177648 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.177241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerDied","Data":"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"}
Sep 12 00:34:44.177700 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.177678    2580 scope.go:117] "RemoveContainer" containerID="4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"
Sep 12 00:34:44.178841 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.178814    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:44.178904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.178886    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:44.178940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:44.178917    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:45.184613 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:45.184576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerStarted","Data":"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"}
Sep 12 00:34:45.740391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:45.740346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:45.740612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:45.740565    2580 patch_prober.go:28] interesting pod/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 12 00:34:45.740688 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:45.740626    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 12 00:34:46.740712 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:46.740673    2580 patch_prober.go:28] interesting pod/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 12 00:34:46.741133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:46.740737    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 12 00:34:47.741292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:47.741254    2580 patch_prober.go:28] interesting pod/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 12 00:34:47.741703 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:47.741311    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 12 00:34:48.740546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:48.740505    2580 patch_prober.go:28] interesting pod/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 12 00:34:48.740727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:48.740567    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 12 00:34:49.741274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:49.741233    2580 patch_prober.go:28] interesting pod/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 12 00:34:49.741735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:49.741327    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 12 00:34:50.744409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:50.744375    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:50.744956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:50.744667    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:50.745333 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:50.745317    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:34:54.712490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.712454    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:34:54.712934 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.712920    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="def2bb2f-c5c8-4507-9114-7d9d0406061d" containerName="istio-proxy"
Sep 12 00:34:54.715488 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.715472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.718490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.718459    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4aaf1a\"/\"autores-model-4aaf1a-kserve-self-signed-certs\""
Sep 12 00:34:54.718490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.718481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4aaf1a\"/\"openshift-service-ca.crt\""
Sep 12 00:34:54.718726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.718536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4aaf1a\"/\"default-dockercfg-q4gfh\""
Sep 12 00:34:54.718726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.718463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4aaf1a\"/\"kube-root-ca.crt\""
Sep 12 00:34:54.726782 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.726740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:34:54.857363 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.857573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.857573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.857573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.857573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.857717 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.857589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjqm2\" (UniqueName: \"kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjqm2\" (UniqueName: \"kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959535 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959629 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.959672 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.959637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.961408 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.961391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.961686 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.961669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:54.966051 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:54.966000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjqm2\" (UniqueName: \"kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2\") pod \"autores-model-4aaf1a-kserve-5659b8cb7-hvf55\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") " pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:55.026029 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.025991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:34:55.191367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.191335    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:55.191594 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.191571    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" podUID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" containerName="maas-api" containerID="cri-o://d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00" gracePeriod=30
Sep 12 00:34:55.237398 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.237303    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:55.237611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.237560    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" podUID="b75e8bef-52ee-41f2-819a-3819ac314813" containerName="payload-processing" containerID="cri-o://519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f" gracePeriod=30
Sep 12 00:34:55.299251 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.299213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:34:55.316668 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:34:55.316642    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf349c7b0_7b55_40bd_9925_62a9dfd6af6b.slice/crio-df7b091e9a9ee959d59c310a56b8a62a9ed93cc10e9f9cd64f80967eb015be33 WatchSource:0}: Error finding container df7b091e9a9ee959d59c310a56b8a62a9ed93cc10e9f9cd64f80967eb015be33: Status 404 returned error can't find the container with id df7b091e9a9ee959d59c310a56b8a62a9ed93cc10e9f9cd64f80967eb015be33
Sep 12 00:34:55.478445 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.478400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:55.529134 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.529055    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:34:55.529283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.529258    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy" containerID="cri-o://f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00" gracePeriod=30
Sep 12 00:34:55.539994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.539966    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:55.566261 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.566225    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs\") pod \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") "
Sep 12 00:34:55.566429 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.566312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") pod \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") "
Sep 12 00:34:55.566429 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.566406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca\") pod \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") "
Sep 12 00:34:55.566533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.566440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65w85\" (UniqueName: \"kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85\") pod \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\" (UID: \"c0602472-6bb2-4f33-8e94-f7613f8ae1d3\") "
Sep 12 00:34:55.566998 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.566966    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0602472-6bb2-4f33-8e94-f7613f8ae1d3" (UID: "c0602472-6bb2-4f33-8e94-f7613f8ae1d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:55.569173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.569144    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85" (OuterVolumeSpecName: "kube-api-access-65w85") pod "c0602472-6bb2-4f33-8e94-f7613f8ae1d3" (UID: "c0602472-6bb2-4f33-8e94-f7613f8ae1d3"). InnerVolumeSpecName "kube-api-access-65w85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:55.570032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.570001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c0602472-6bb2-4f33-8e94-f7613f8ae1d3" (UID: "c0602472-6bb2-4f33-8e94-f7613f8ae1d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:55.570711 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.570684    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0602472-6bb2-4f33-8e94-f7613f8ae1d3" (UID: "c0602472-6bb2-4f33-8e94-f7613f8ae1d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:55.667619 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf4g9\" (UniqueName: \"kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9\") pod \"b75e8bef-52ee-41f2-819a-3819ac314813\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") "
Sep 12 00:34:55.667823 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume\") pod \"b75e8bef-52ee-41f2-819a-3819ac314813\" (UID: \"b75e8bef-52ee-41f2-819a-3819ac314813\") "
Sep 12 00:34:55.667906 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667855    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-openshift-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.667906 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667867    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65w85\" (UniqueName: \"kubernetes.io/projected/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-kube-api-access-65w85\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.667906 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667876    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-metrics-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.667906 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.667885    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0602472-6bb2-4f33-8e94-f7613f8ae1d3-maas-api-tls\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.668137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.668108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b75e8bef-52ee-41f2-819a-3819ac314813" (UID: "b75e8bef-52ee-41f2-819a-3819ac314813"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:55.669623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.669603    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9" (OuterVolumeSpecName: "kube-api-access-zf4g9") pod "b75e8bef-52ee-41f2-819a-3819ac314813" (UID: "b75e8bef-52ee-41f2-819a-3819ac314813"). InnerVolumeSpecName "kube-api-access-zf4g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:55.768569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.768528    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b75e8bef-52ee-41f2-819a-3819ac314813-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.768569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:55.768562    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf4g9\" (UniqueName: \"kubernetes.io/projected/b75e8bef-52ee-41f2-819a-3819ac314813-kube-api-access-zf4g9\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:34:56.246925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.246842    2580 generic.go:356] "Generic (PLEG): container finished" podID="b75e8bef-52ee-41f2-819a-3819ac314813" containerID="519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f" exitCode=0
Sep 12 00:34:56.246925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.246902    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"
Sep 12 00:34:56.247134 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.246930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" event={"ID":"b75e8bef-52ee-41f2-819a-3819ac314813","Type":"ContainerDied","Data":"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"}
Sep 12 00:34:56.247134 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.247019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r" event={"ID":"b75e8bef-52ee-41f2-819a-3819ac314813","Type":"ContainerDied","Data":"070a693441352e523e83f954dc9f324157de1fb939e84dd61141a9aedc1df6c5"}
Sep 12 00:34:56.247134 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.247055    2580 scope.go:117] "RemoveContainer" containerID="519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"
Sep 12 00:34:56.248417 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.248395    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" containerID="d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00" exitCode=0
Sep 12 00:34:56.248498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.248449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"
Sep 12 00:34:56.248559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.248478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" event={"ID":"c0602472-6bb2-4f33-8e94-f7613f8ae1d3","Type":"ContainerDied","Data":"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"}
Sep 12 00:34:56.248559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.248547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624" event={"ID":"c0602472-6bb2-4f33-8e94-f7613f8ae1d3","Type":"ContainerDied","Data":"584c1aa831f66b8107c7a0f6ff0a26de52f2a954c54cba55dfb188b1fab5fb21"}
Sep 12 00:34:56.250522 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.250495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" event={"ID":"f349c7b0-7b55-40bd-9925-62a9dfd6af6b","Type":"ContainerStarted","Data":"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"}
Sep 12 00:34:56.250660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.250536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" event={"ID":"f349c7b0-7b55-40bd-9925-62a9dfd6af6b","Type":"ContainerStarted","Data":"df7b091e9a9ee959d59c310a56b8a62a9ed93cc10e9f9cd64f80967eb015be33"}
Sep 12 00:34:56.258273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.258258    2580 scope.go:117] "RemoveContainer" containerID="519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"
Sep 12 00:34:56.258552 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:56.258534    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f\": container with ID starting with 519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f not found: ID does not exist" containerID="519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"
Sep 12 00:34:56.258609 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.258562    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f"} err="failed to get container status \"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f\": rpc error: code = NotFound desc = could not find container \"519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f\": container with ID starting with 519b7c20b49806d47f36f31c903e05566646511f8ffd0a6343d89af58584578f not found: ID does not exist"
Sep 12 00:34:56.258609 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.258579    2580 scope.go:117] "RemoveContainer" containerID="d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"
Sep 12 00:34:56.267744 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.267723    2580 scope.go:117] "RemoveContainer" containerID="d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"
Sep 12 00:34:56.267978 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.267941    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" podStartSLOduration=2.267927602 podStartE2EDuration="2.267927602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:56.264818117 +0000 UTC m=+1092.398808343" watchObservedRunningTime="2026-09-12 00:34:56.267927602 +0000 UTC m=+1092.401917831"
Sep 12 00:34:56.268069 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:34:56.268052    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00\": container with ID starting with d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00 not found: ID does not exist" containerID="d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"
Sep 12 00:34:56.268114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.268076    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00"} err="failed to get container status \"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00\": rpc error: code = NotFound desc = could not find container \"d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00\": container with ID starting with d546ddfdd51be0cc1f890cd65e072713028cb2bef8057c9ce31e7179106b9f00 not found: ID does not exist"
Sep 12 00:34:56.282754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.282726    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:56.292044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.292021    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6d0c6f2-f84d5d6b8-5g624"]
Sep 12 00:34:56.303865 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.303836    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:56.317089 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.317063    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e6d0c6f2-555447fffd-mpd5r"]
Sep 12 00:34:56.328538 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328512    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:34:56.328905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328888    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75e8bef-52ee-41f2-819a-3819ac314813" containerName="payload-processing"
Sep 12 00:34:56.328905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328903    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75e8bef-52ee-41f2-819a-3819ac314813" containerName="payload-processing"
Sep 12 00:34:56.329046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328918    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" containerName="maas-api"
Sep 12 00:34:56.329046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328923    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" containerName="maas-api"
Sep 12 00:34:56.329046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328991    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75e8bef-52ee-41f2-819a-3819ac314813" containerName="payload-processing"
Sep 12 00:34:56.329046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.328997    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" containerName="maas-api"
Sep 12 00:34:56.331667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.331645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.333776 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.333744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6cbdb29e-openshift-default-dockercfg-7vzhw\""
Sep 12 00:34:56.344052 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.344026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:34:56.374039 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.374010    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b75e8bef-52ee-41f2-819a-3819ac314813" path="/var/lib/kubelet/pods/b75e8bef-52ee-41f2-819a-3819ac314813/volumes"
Sep 12 00:34:56.374499 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.374481    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0602472-6bb2-4f33-8e94-f7613f8ae1d3" path="/var/lib/kubelet/pods/c0602472-6bb2-4f33-8e94-f7613f8ae1d3/volumes"
Sep 12 00:34:56.475291 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4njz\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475291 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.475908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.475649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4njz\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.576932 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.576939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.577162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.577353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.577405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.577487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.577529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.577487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.579037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.579013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.579280 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.579261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.585201 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.585181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.585397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.585379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4njz\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz\") pod \"e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") " pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.642593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.642557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:56.920194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.920169    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:34:56.921644 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:34:56.921616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddb182c_4dcd_407c_a3c8_7dfd0b3c0ea7.slice/crio-986104c854694f1c2a7fc58fb43e536cb2e6ada39c618ba3358e358a0f606757 WatchSource:0}: Error finding container 986104c854694f1c2a7fc58fb43e536cb2e6ada39c618ba3358e358a0f606757: Status 404 returned error can't find the container with id 986104c854694f1c2a7fc58fb43e536cb2e6ada39c618ba3358e358a0f606757
Sep 12 00:34:56.925118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.925082    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:56.925219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.925154    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:56.925219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:56.925193    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:34:57.261754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:57.261712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerStarted","Data":"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"}
Sep 12 00:34:57.261975 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:57.261785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerStarted","Data":"986104c854694f1c2a7fc58fb43e536cb2e6ada39c618ba3358e358a0f606757"}
Sep 12 00:34:57.279694 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:57.279640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podStartSLOduration=1.279622821 podStartE2EDuration="1.279622821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:57.278682704 +0000 UTC m=+1093.412672929" watchObservedRunningTime="2026-09-12 00:34:57.279622821 +0000 UTC m=+1093.413613047"
Sep 12 00:34:57.643109 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:57.643020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:57.648051 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:57.648026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:58.269640 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:58.269597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:34:58.270808 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:34:58.270760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:00.614503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.614467    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:00.616856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.616838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.618860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.618840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6cbdb29e-serving-cert\""
Sep 12 00:35:00.626729 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.626705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:00.718573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.718542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnszl\" (UniqueName: \"kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.718724 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.718597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.718724 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.718671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.718842 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.718740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.781376 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.781349    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:35:00.820256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.820168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.820256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.820219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.820478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.820305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnszl\" (UniqueName: \"kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.820478 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:00.820316    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-serving-cert: secret "maas-api-e2e-ait-6cbdb29e-serving-cert" not found
Sep 12 00:35:00.820478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.820356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.820478 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:00.820378    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls podName:5db96549-a3e9-4698-9458-42c53506d31f nodeName:}" failed. No retries permitted until 2026-09-12 00:35:01.320360917 +0000 UTC m=+1097.454351138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls") pod "maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" (UID: "5db96549-a3e9-4698-9458-42c53506d31f") : secret "maas-api-e2e-ait-6cbdb29e-serving-cert" not found
Sep 12 00:35:00.821021 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.820999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.822725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.822703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.828074 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.828049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnszl\" (UniqueName: \"kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:00.921552 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921519    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921633    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921701    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.921739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.922157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbpwq\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq\") pod \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\" (UID: \"0536a0ce-03c6-4b39-9b6d-e088d310de1c\") "
Sep 12 00:35:00.922157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:00.922157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.921945    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:00.922157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922152    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:00.922358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:00.922358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:00.922358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922281    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:00.922358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922301    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:00.922358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.922316    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:00.923979 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.923948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:00.924083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.923991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:00.924083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.923999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:35:00.924083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:00.924020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq" (OuterVolumeSpecName: "kube-api-access-cbpwq") pod "0536a0ce-03c6-4b39-9b6d-e088d310de1c" (UID: "0536a0ce-03c6-4b39-9b6d-e088d310de1c"). InnerVolumeSpecName "kube-api-access-cbpwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:01.022908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022877    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.022908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022907    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.022908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022918    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.023147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022928    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbpwq\" (UniqueName: \"kubernetes.io/projected/0536a0ce-03c6-4b39-9b6d-e088d310de1c-kube-api-access-cbpwq\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.023147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022937    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.023147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.022946    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0536a0ce-03c6-4b39-9b6d-e088d310de1c-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:01.264829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.264787    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:01.265252 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.265232    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.265346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.265254    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.265399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.265381    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.265399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.265396    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.267994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.267972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.270169 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.270152    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6cbdb29e-dockercfg-5k66c\""
Sep 12 00:35:01.270260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.270153    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6cbdb29e\""
Sep 12 00:35:01.278286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.278265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:01.289489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.289323    2580 generic.go:356] "Generic (PLEG): container finished" podID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerID="f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00" exitCode=0
Sep 12 00:35:01.289489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.289381    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"
Sep 12 00:35:01.289489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.289413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerDied","Data":"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"}
Sep 12 00:35:01.289489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.289488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z" event={"ID":"0536a0ce-03c6-4b39-9b6d-e088d310de1c","Type":"ContainerDied","Data":"d32b26541084d100f15b66cf9828e5b55c5ab16976903e992502bd5a49504219"}
Sep 12 00:35:01.289853 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.289531    2580 scope.go:117] "RemoveContainer" containerID="f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"
Sep 12 00:35:01.299492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.299466    2580 scope.go:117] "RemoveContainer" containerID="4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"
Sep 12 00:35:01.308062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.308042    2580 scope.go:117] "RemoveContainer" containerID="f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"
Sep 12 00:35:01.308346 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:01.308323    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00\": container with ID starting with f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00 not found: ID does not exist" containerID="f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"
Sep 12 00:35:01.308426 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.308359    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00"} err="failed to get container status \"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00\": rpc error: code = NotFound desc = could not find container \"f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00\": container with ID starting with f3fef3472ed0814f88af8d44075bd9fcb47285558383ed7de0d8b8b19d4eba00 not found: ID does not exist"
Sep 12 00:35:01.308426 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.308384    2580 scope.go:117] "RemoveContainer" containerID="4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"
Sep 12 00:35:01.308657 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:01.308638    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043\": container with ID starting with 4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043 not found: ID does not exist" containerID="4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"
Sep 12 00:35:01.308697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.308663    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043"} err="failed to get container status \"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043\": rpc error: code = NotFound desc = could not find container \"4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043\": container with ID starting with 4012b17b92d2744ca7da8bfaa7b3c52ecacd6cc2b351db2c30218a94309a2043 not found: ID does not exist"
Sep 12 00:35:01.314815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.314757    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:01.315198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.315182    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.315198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.315199    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" containerName="istio-proxy"
Sep 12 00:35:01.317805 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.317761    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:35:01.317963 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.317879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.326611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.326519    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-e6d0c6f2-gw-openshift-default-64fc5dd48d-z6r8z"]
Sep 12 00:35:01.328442 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.328410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.328587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.328475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.328587 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.328519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56745\" (UniqueName: \"kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.328713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.328628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:01.328713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.328675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxscc\" (UniqueName: \"kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.329669 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.329640    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:01.331755 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.331730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") pod \"maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:01.429831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.429761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.430012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.429855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.430012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.429890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56745\" (UniqueName: \"kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.430012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.429958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxscc\" (UniqueName: \"kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.430537 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.430515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.430630 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.430609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.437583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.437562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxscc\" (UniqueName: \"kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc\") pod \"payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.437696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.437618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56745\" (UniqueName: \"kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745\") pod \"payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") " pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.528064 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.527972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:01.580201 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.580174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:01.636434 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.636398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:01.803798 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.803752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:01.806182 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:01.806156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db96549_a3e9_4698_9458_42c53506d31f.slice/crio-8907e5f66edd358e5ce3d99dfd07230d2af8ce7bd7f619667db90c59e611e7e1 WatchSource:0}: Error finding container 8907e5f66edd358e5ce3d99dfd07230d2af8ce7bd7f619667db90c59e611e7e1: Status 404 returned error can't find the container with id 8907e5f66edd358e5ce3d99dfd07230d2af8ce7bd7f619667db90c59e611e7e1
Sep 12 00:35:01.870267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.870235    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:01.872018 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:01.871986    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ac40d9_ef4e_4595_8d78_967604580e16.slice/crio-4c00d5665d3d5074fd76bde688f0d5698bfd9f85354bc91d722b6f084280c62c WatchSource:0}: Error finding container 4c00d5665d3d5074fd76bde688f0d5698bfd9f85354bc91d722b6f084280c62c: Status 404 returned error can't find the container with id 4c00d5665d3d5074fd76bde688f0d5698bfd9f85354bc91d722b6f084280c62c
Sep 12 00:35:01.922964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:01.922939    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:01.924977 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:01.924940    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1ce920_9975_4c44_9bc2_24ce6eec5971.slice/crio-d30cf5dec17936a35dcd80979e4ffe82375baefb4adcd2dc6a7b01f3c50cd9c3 WatchSource:0}: Error finding container d30cf5dec17936a35dcd80979e4ffe82375baefb4adcd2dc6a7b01f3c50cd9c3: Status 404 returned error can't find the container with id d30cf5dec17936a35dcd80979e4ffe82375baefb4adcd2dc6a7b01f3c50cd9c3
Sep 12 00:35:02.300032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.299990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" event={"ID":"eb1ce920-9975-4c44-9bc2-24ce6eec5971","Type":"ContainerStarted","Data":"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"}
Sep 12 00:35:02.300942 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.300793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:02.300942 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.300838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" event={"ID":"eb1ce920-9975-4c44-9bc2-24ce6eec5971","Type":"ContainerStarted","Data":"d30cf5dec17936a35dcd80979e4ffe82375baefb4adcd2dc6a7b01f3c50cd9c3"}
Sep 12 00:35:02.302107 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.302027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" event={"ID":"5db96549-a3e9-4698-9458-42c53506d31f","Type":"ContainerStarted","Data":"8907e5f66edd358e5ce3d99dfd07230d2af8ce7bd7f619667db90c59e611e7e1"}
Sep 12 00:35:02.304126 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.304104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" event={"ID":"37ac40d9-ef4e-4595-8d78-967604580e16","Type":"ContainerStarted","Data":"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"}
Sep 12 00:35:02.304239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.304150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" event={"ID":"37ac40d9-ef4e-4595-8d78-967604580e16","Type":"ContainerStarted","Data":"4c00d5665d3d5074fd76bde688f0d5698bfd9f85354bc91d722b6f084280c62c"}
Sep 12 00:35:02.304239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.304211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:02.318005 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.317953    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" podStartSLOduration=1.317932429 podStartE2EDuration="1.317932429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:02.315544371 +0000 UTC m=+1098.449534594" watchObservedRunningTime="2026-09-12 00:35:02.317932429 +0000 UTC m=+1098.451922652"
Sep 12 00:35:02.330038 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.329976    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" podStartSLOduration=1.329955254 podStartE2EDuration="1.329955254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:02.328985475 +0000 UTC m=+1098.462975704" watchObservedRunningTime="2026-09-12 00:35:02.329955254 +0000 UTC m=+1098.463945481"
Sep 12 00:35:02.374337 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:02.374302    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0536a0ce-03c6-4b39-9b6d-e088d310de1c" path="/var/lib/kubelet/pods/0536a0ce-03c6-4b39-9b6d-e088d310de1c/volumes"
Sep 12 00:35:03.311582 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:03.311539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" event={"ID":"5db96549-a3e9-4698-9458-42c53506d31f","Type":"ContainerStarted","Data":"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"}
Sep 12 00:35:03.312079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:03.311956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:03.327697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:03.327634    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" podStartSLOduration=2.587429664 podStartE2EDuration="3.327613588s" totalImagesPullingTime="740.183924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:03.326788701 +0000 UTC m=+1099.460778920" watchObservedRunningTime="2026-09-12 00:35:03.327613588 +0000 UTC m=+1099.461603815"
Sep 12 00:35:05.027175 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:05.027121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:05.027175 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:05.027191    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:05.039909 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:05.039881    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:05.335934 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:05.335848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:09.333172 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:09.333130    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:13.313637 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:13.313604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:13.314031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:13.313679    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:14.311947 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:14.311899    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 12 00:35:14.312138 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:14.311990    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 12 00:35:16.406964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.406926    2580 generic.go:356] "Generic (PLEG): container finished" podID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerID="a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64" exitCode=0
Sep 12 00:35:16.407390 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.406997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerDied","Data":"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"}
Sep 12 00:35:16.407434 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.407394    2580 scope.go:117] "RemoveContainer" containerID="a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"
Sep 12 00:35:16.408542 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.408513    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:35:16.408600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.408589    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:35:16.408636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:16.408622    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:35:17.414936 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:17.414897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerStarted","Data":"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"}
Sep 12 00:35:17.643471 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:17.643430    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:17.643728 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:17.643703    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:17.643788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:17.643752    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:18.643054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:18.643005    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:18.643576 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:18.643061    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:19.643573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:19.643532    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:19.643964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:19.643593    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:20.643739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:20.643694    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:20.644247 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:20.643802    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:21.643602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:21.643564    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:21.643799 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:21.643646    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:22.643831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:22.643783    2580 patch_prober.go:28] interesting pod/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:35:22.644226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:22.643849    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 12 00:35:23.646729 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:23.646692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:23.647121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:23.646975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:23.647545 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:23.647530    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:25.616055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.616017    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:35:25.618703 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.618684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.621530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.621491    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a4df66\"/\"kube-root-ca.crt\""
Sep 12 00:35:25.621530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.621493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a4df66\"/\"default-dockercfg-hrkp4\""
Sep 12 00:35:25.621714 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.621493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a4df66\"/\"test-model-a4df66-kserve-self-signed-certs\""
Sep 12 00:35:25.621714 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.621542    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a4df66\"/\"openshift-service-ca.crt\""
Sep 12 00:35:25.629824 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.629801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:35:25.791798 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.791749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.792020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.791902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.792020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.791973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.792020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.792014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.792217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.792084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pk4w\" (UniqueName: \"kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.792217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.792150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.892937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.892833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.892937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.892897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.892937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.892924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.892937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.892948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pk4w\" (UniqueName: \"kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.893285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.892983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.893285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.893059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.893407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.893382    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.893462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.893412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.893462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.893439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.895089 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.895067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.895431 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.895414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.899578 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.899560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pk4w\" (UniqueName: \"kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w\") pod \"test-model-a4df66-kserve-74bbbb47f5-wmnls\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") " pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:25.930994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:25.930957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:26.205834 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:26.205806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:35:26.207230 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:26.207205    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod586e7728_8437_49fd_a92b_6b425c115945.slice/crio-5c0a989c90116aa3bd449ed947f2791a0853fe9fa00ec2689a28514cb4ed249f WatchSource:0}: Error finding container 5c0a989c90116aa3bd449ed947f2791a0853fe9fa00ec2689a28514cb4ed249f: Status 404 returned error can't find the container with id 5c0a989c90116aa3bd449ed947f2791a0853fe9fa00ec2689a28514cb4ed249f
Sep 12 00:35:26.480131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:26.480035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" event={"ID":"586e7728-8437-49fd-a92b-6b425c115945","Type":"ContainerStarted","Data":"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"}
Sep 12 00:35:26.480131 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:26.480083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" event={"ID":"586e7728-8437-49fd-a92b-6b425c115945","Type":"ContainerStarted","Data":"5c0a989c90116aa3bd449ed947f2791a0853fe9fa00ec2689a28514cb4ed249f"}
Sep 12 00:35:26.495079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:26.495014    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" podStartSLOduration=1.494995309 podStartE2EDuration="1.494995309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:26.494544417 +0000 UTC m=+1122.628534642" watchObservedRunningTime="2026-09-12 00:35:26.494995309 +0000 UTC m=+1122.628985537"
Sep 12 00:35:31.737885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.737846    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:31.738277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.738115    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" podUID="5db96549-a3e9-4698-9458-42c53506d31f" containerName="maas-api" containerID="cri-o://70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa" gracePeriod=30
Sep 12 00:35:31.813526 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.813487    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:31.813749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.813718    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" podUID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" containerName="payload-processing" containerID="cri-o://2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052" gracePeriod=30
Sep 12 00:35:31.820656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.820626    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:31.820927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:31.820870    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" podUID="37ac40d9-ef4e-4595-8d78-967604580e16" containerName="payload-pre-processing" containerID="cri-o://be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c" gracePeriod=30
Sep 12 00:35:32.060563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.060536    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:32.110258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.107687    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:32.110258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.109369    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:32.159811 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.159756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs\") pod \"5db96549-a3e9-4698-9458-42c53506d31f\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") "
Sep 12 00:35:32.159987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.159824    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") pod \"5db96549-a3e9-4698-9458-42c53506d31f\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") "
Sep 12 00:35:32.159987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.159854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnszl\" (UniqueName: \"kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl\") pod \"5db96549-a3e9-4698-9458-42c53506d31f\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") "
Sep 12 00:35:32.159987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.159943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca\") pod \"5db96549-a3e9-4698-9458-42c53506d31f\" (UID: \"5db96549-a3e9-4698-9458-42c53506d31f\") "
Sep 12 00:35:32.160347 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.160320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5db96549-a3e9-4698-9458-42c53506d31f" (UID: "5db96549-a3e9-4698-9458-42c53506d31f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:32.162019 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.161991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5db96549-a3e9-4698-9458-42c53506d31f" (UID: "5db96549-a3e9-4698-9458-42c53506d31f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:32.162275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.162261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5db96549-a3e9-4698-9458-42c53506d31f" (UID: "5db96549-a3e9-4698-9458-42c53506d31f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:32.162391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.162376    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl" (OuterVolumeSpecName: "kube-api-access-dnszl") pod "5db96549-a3e9-4698-9458-42c53506d31f" (UID: "5db96549-a3e9-4698-9458-42c53506d31f"). InnerVolumeSpecName "kube-api-access-dnszl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:32.261463 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.261360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume\") pod \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") "
Sep 12 00:35:32.261463 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.261445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxscc\" (UniqueName: \"kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc\") pod \"37ac40d9-ef4e-4595-8d78-967604580e16\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") "
Sep 12 00:35:32.261722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.261500    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56745\" (UniqueName: \"kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745\") pod \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\" (UID: \"eb1ce920-9975-4c44-9bc2-24ce6eec5971\") "
Sep 12 00:35:32.261722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.261562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume\") pod \"37ac40d9-ef4e-4595-8d78-967604580e16\" (UID: \"37ac40d9-ef4e-4595-8d78-967604580e16\") "
Sep 12 00:35:32.261896 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.261858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb1ce920-9975-4c44-9bc2-24ce6eec5971" (UID: "eb1ce920-9975-4c44-9bc2-24ce6eec5971"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:32.262025 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262005    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-metrics-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.262117 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262030    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1ce920-9975-4c44-9bc2-24ce6eec5971-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.262117 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262043    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db96549-a3e9-4698-9458-42c53506d31f-maas-api-tls\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.262117 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262057    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnszl\" (UniqueName: \"kubernetes.io/projected/5db96549-a3e9-4698-9458-42c53506d31f-kube-api-access-dnszl\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.262117 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262069    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db96549-a3e9-4698-9458-42c53506d31f-openshift-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.262315 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.262191    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37ac40d9-ef4e-4595-8d78-967604580e16" (UID: "37ac40d9-ef4e-4595-8d78-967604580e16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:32.263786 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.263744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc" (OuterVolumeSpecName: "kube-api-access-bxscc") pod "37ac40d9-ef4e-4595-8d78-967604580e16" (UID: "37ac40d9-ef4e-4595-8d78-967604580e16"). InnerVolumeSpecName "kube-api-access-bxscc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:32.263964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.263945    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745" (OuterVolumeSpecName: "kube-api-access-56745") pod "eb1ce920-9975-4c44-9bc2-24ce6eec5971" (UID: "eb1ce920-9975-4c44-9bc2-24ce6eec5971"). InnerVolumeSpecName "kube-api-access-56745". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:32.363883 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.363755    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxscc\" (UniqueName: \"kubernetes.io/projected/37ac40d9-ef4e-4595-8d78-967604580e16-kube-api-access-bxscc\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.363883 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.363817    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56745\" (UniqueName: \"kubernetes.io/projected/eb1ce920-9975-4c44-9bc2-24ce6eec5971-kube-api-access-56745\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.363883 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.363832    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37ac40d9-ef4e-4595-8d78-967604580e16-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:32.518017 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.517922    2580 generic.go:356] "Generic (PLEG): container finished" podID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" containerID="2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052" exitCode=0
Sep 12 00:35:32.518017 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.517969    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"
Sep 12 00:35:32.518246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.518010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" event={"ID":"eb1ce920-9975-4c44-9bc2-24ce6eec5971","Type":"ContainerDied","Data":"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"}
Sep 12 00:35:32.518246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.518063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw" event={"ID":"eb1ce920-9975-4c44-9bc2-24ce6eec5971","Type":"ContainerDied","Data":"d30cf5dec17936a35dcd80979e4ffe82375baefb4adcd2dc6a7b01f3c50cd9c3"}
Sep 12 00:35:32.518246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.518092    2580 scope.go:117] "RemoveContainer" containerID="2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"
Sep 12 00:35:32.519783 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.519739    2580 generic.go:356] "Generic (PLEG): container finished" podID="5db96549-a3e9-4698-9458-42c53506d31f" containerID="70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa" exitCode=0
Sep 12 00:35:32.519912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.519787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" event={"ID":"5db96549-a3e9-4698-9458-42c53506d31f","Type":"ContainerDied","Data":"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"}
Sep 12 00:35:32.519912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.519816    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"
Sep 12 00:35:32.519912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.519836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4" event={"ID":"5db96549-a3e9-4698-9458-42c53506d31f","Type":"ContainerDied","Data":"8907e5f66edd358e5ce3d99dfd07230d2af8ce7bd7f619667db90c59e611e7e1"}
Sep 12 00:35:32.521959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.521936    2580 generic.go:356] "Generic (PLEG): container finished" podID="37ac40d9-ef4e-4595-8d78-967604580e16" containerID="be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c" exitCode=0
Sep 12 00:35:32.522090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.521967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" event={"ID":"37ac40d9-ef4e-4595-8d78-967604580e16","Type":"ContainerDied","Data":"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"}
Sep 12 00:35:32.522090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.521987    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"
Sep 12 00:35:32.522209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.522002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv" event={"ID":"37ac40d9-ef4e-4595-8d78-967604580e16","Type":"ContainerDied","Data":"4c00d5665d3d5074fd76bde688f0d5698bfd9f85354bc91d722b6f084280c62c"}
Sep 12 00:35:32.529597 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.529577    2580 scope.go:117] "RemoveContainer" containerID="2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"
Sep 12 00:35:32.529966 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:32.529947    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052\": container with ID starting with 2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052 not found: ID does not exist" containerID="2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"
Sep 12 00:35:32.530062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.529979    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052"} err="failed to get container status \"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052\": rpc error: code = NotFound desc = could not find container \"2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052\": container with ID starting with 2e95004878ddbe9646d51342b2280b5a2e045b1faeca0ac78ff9f557332f0052 not found: ID does not exist"
Sep 12 00:35:32.530062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.530001    2580 scope.go:117] "RemoveContainer" containerID="70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"
Sep 12 00:35:32.540599 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.540576    2580 scope.go:117] "RemoveContainer" containerID="70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"
Sep 12 00:35:32.540941 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:32.540918    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa\": container with ID starting with 70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa not found: ID does not exist" containerID="70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"
Sep 12 00:35:32.540996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.540953    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa"} err="failed to get container status \"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa\": rpc error: code = NotFound desc = could not find container \"70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa\": container with ID starting with 70c28618a2f0dfab7998bbc0436bfd14f80e82a5edb782011c6650eccc46ddfa not found: ID does not exist"
Sep 12 00:35:32.540996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.540974    2580 scope.go:117] "RemoveContainer" containerID="be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"
Sep 12 00:35:32.541309 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.541284    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:32.547408 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.547377    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cbdb29e-5ff6769957-whtjw"]
Sep 12 00:35:32.549874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.549851    2580 scope.go:117] "RemoveContainer" containerID="be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"
Sep 12 00:35:32.550230 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:32.550207    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c\": container with ID starting with be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c not found: ID does not exist" containerID="be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"
Sep 12 00:35:32.550343 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.550236    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c"} err="failed to get container status \"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c\": rpc error: code = NotFound desc = could not find container \"be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c\": container with ID starting with be1a70510ae76409ce3c5f086926e0da9663a04eb62514c442402c5057d3622c not found: ID does not exist"
Sep 12 00:35:32.564555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.564523    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:32.567878 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.567851    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6cbdb29e-6b8974b958-868qv"]
Sep 12 00:35:32.578161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.578136    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:32.585009 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:32.584978    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cbdb29e-85d4d5d9f6-qfwj4"]
Sep 12 00:35:33.199043 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:33.199005    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:35:33.199551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:33.199338    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy" containerID="cri-o://9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1" gracePeriod=30
Sep 12 00:35:34.374655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:34.374608    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ac40d9-ef4e-4595-8d78-967604580e16" path="/var/lib/kubelet/pods/37ac40d9-ef4e-4595-8d78-967604580e16/volumes"
Sep 12 00:35:34.375234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:34.375213    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db96549-a3e9-4698-9458-42c53506d31f" path="/var/lib/kubelet/pods/5db96549-a3e9-4698-9458-42c53506d31f/volumes"
Sep 12 00:35:34.375611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:34.375592    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" path="/var/lib/kubelet/pods/eb1ce920-9975-4c44-9bc2-24ce6eec5971/volumes"
Sep 12 00:35:35.931642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:35.931608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:35.932114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:35.931885    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:35.944351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:35.944322    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:36.557259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:36.557232    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:35:38.453970 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.453943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:38.521001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.520913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4njz\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.520962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521245 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521245 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521245 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521203    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521245 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521242    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521350    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket\") pod \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\" (UID: \"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7\") "
Sep 12 00:35:38.521456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:38.521665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521460    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:38.521665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:38.521665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:38.521864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521679    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:38.521864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521829    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.521864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521847    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.521864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521861    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.522075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521874    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.522075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.521888    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.523250 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.523222    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:38.523250 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.523240    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz" (OuterVolumeSpecName: "kube-api-access-d4njz") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "kube-api-access-d4njz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:38.523394 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.523257    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:35:38.523394 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.523389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" (UID: "6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:38.559227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.559200    2580 generic.go:356] "Generic (PLEG): container finished" podID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerID="9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1" exitCode=0
Sep 12 00:35:38.559366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.559245    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"
Sep 12 00:35:38.559366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.559263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerDied","Data":"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"}
Sep 12 00:35:38.559366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.559314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk" event={"ID":"6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7","Type":"ContainerDied","Data":"986104c854694f1c2a7fc58fb43e536cb2e6ada39c618ba3358e358a0f606757"}
Sep 12 00:35:38.559366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.559337    2580 scope.go:117] "RemoveContainer" containerID="9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"
Sep 12 00:35:38.568649 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.568630    2580 scope.go:117] "RemoveContainer" containerID="a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"
Sep 12 00:35:38.577310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.577293    2580 scope.go:117] "RemoveContainer" containerID="9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"
Sep 12 00:35:38.577579 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:38.577562    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1\": container with ID starting with 9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1 not found: ID does not exist" containerID="9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"
Sep 12 00:35:38.577631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.577585    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1"} err="failed to get container status \"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1\": rpc error: code = NotFound desc = could not find container \"9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1\": container with ID starting with 9848627d1420f5ab0702ffd499e2cdc88202172481bca467f7238037d631c7e1 not found: ID does not exist"
Sep 12 00:35:38.577631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.577601    2580 scope.go:117] "RemoveContainer" containerID="a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"
Sep 12 00:35:38.577851 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:38.577834    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64\": container with ID starting with a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64 not found: ID does not exist" containerID="a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"
Sep 12 00:35:38.577919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.577870    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64"} err="failed to get container status \"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64\": rpc error: code = NotFound desc = could not find container \"a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64\": container with ID starting with a0b7fee60cc09fca646dda3f7eefab7583351bede950f5eab3426611efb79c64 not found: ID does not exist"
Sep 12 00:35:38.581572 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.581551    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:35:38.584022 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.584000    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6cbdb29e-openshift-default-869ccf6ffd-w7ksk"]
Sep 12 00:35:38.623037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.623005    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4njz\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-kube-api-access-d4njz\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.623037 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.623033    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.623217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.623048    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.623217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:38.623061    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:40.374890 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:40.374857    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" path="/var/lib/kubelet/pods/6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7/volumes"
Sep 12 00:35:41.616228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616194    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616565    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" containerName="payload-processing"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616576    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" containerName="payload-processing"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616584    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616590    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616598    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db96549-a3e9-4698-9458-42c53506d31f" containerName="maas-api"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616604    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db96549-a3e9-4698-9458-42c53506d31f" containerName="maas-api"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616611    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.616612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616616    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616626    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ac40d9-ef4e-4595-8d78-967604580e16" containerName="payload-pre-processing"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ac40d9-ef4e-4595-8d78-967604580e16" containerName="payload-pre-processing"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616686    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ac40d9-ef4e-4595-8d78-967604580e16" containerName="payload-pre-processing"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616696    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1ce920-9975-4c44-9bc2-24ce6eec5971" containerName="payload-processing"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616702    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616709    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db96549-a3e9-4698-9458-42c53506d31f" containerName="maas-api"
Sep 12 00:35:41.616886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.616714    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ddb182c-4dcd-407c-a3c8-7dfd0b3c0ea7" containerName="istio-proxy"
Sep 12 00:35:41.619372 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.619357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.621457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.621437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9c414efe-dockercfg-6z6zw\""
Sep 12 00:35:41.621561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.621463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9c414efe\""
Sep 12 00:35:41.630349 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.630328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:35:41.639726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.639702    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:35:41.641974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.641957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.651712 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.651681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlhz6\" (UniqueName: \"kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.651872 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.651729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.654295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.654268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:35:41.752981 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.752948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.753158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.752989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgdzs\" (UniqueName: \"kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.753158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.753125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.753239 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.753177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlhz6\" (UniqueName: \"kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.753817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.753797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.763874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.763843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlhz6\" (UniqueName: \"kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6\") pod \"payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.854170 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.854128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.854317 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.854195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdzs\" (UniqueName: \"kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.854841 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.854822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.861956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.861931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgdzs\" (UniqueName: \"kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs\") pod \"payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") " pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:41.930595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.930563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:41.952017 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:41.951990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:42.209696 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.209672    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:35:42.211102 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:42.211072    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a2eee8_04b2_4f74_94fb_08b6b3d5cca9.slice/crio-fdd7fd0f97cf13de8e550f5b07548917fcf9150aedfe8ca9811bfaecb1c50c99 WatchSource:0}: Error finding container fdd7fd0f97cf13de8e550f5b07548917fcf9150aedfe8ca9811bfaecb1c50c99: Status 404 returned error can't find the container with id fdd7fd0f97cf13de8e550f5b07548917fcf9150aedfe8ca9811bfaecb1c50c99
Sep 12 00:35:42.258451 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.258423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:35:42.259645 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:35:42.259616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dcc666_3cff_4ead_8dba_44bd11408651.slice/crio-93501bec72e48e967a46604c343233fb04c9cdc6b280e063d7bbfe9a1dc4b46c WatchSource:0}: Error finding container 93501bec72e48e967a46604c343233fb04c9cdc6b280e063d7bbfe9a1dc4b46c: Status 404 returned error can't find the container with id 93501bec72e48e967a46604c343233fb04c9cdc6b280e063d7bbfe9a1dc4b46c
Sep 12 00:35:42.584777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.584667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" event={"ID":"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9","Type":"ContainerStarted","Data":"53a08f721e48530a8a1086cd1345c83d5084b524bde0921ee480f171467d2dba"}
Sep 12 00:35:42.584777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.584729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" event={"ID":"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9","Type":"ContainerStarted","Data":"fdd7fd0f97cf13de8e550f5b07548917fcf9150aedfe8ca9811bfaecb1c50c99"}
Sep 12 00:35:42.584777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.584756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:42.586425 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.586394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" event={"ID":"72dcc666-3cff-4ead-8dba-44bd11408651","Type":"ContainerStarted","Data":"1a987a5e28dae374daaad5b7f91b6106f29d52257bcd15ba283b0bd9fd4c41c7"}
Sep 12 00:35:42.586556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.586440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" event={"ID":"72dcc666-3cff-4ead-8dba-44bd11408651","Type":"ContainerStarted","Data":"93501bec72e48e967a46604c343233fb04c9cdc6b280e063d7bbfe9a1dc4b46c"}
Sep 12 00:35:42.586556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.586488    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:42.600911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.600859    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" podStartSLOduration=1.600843664 podStartE2EDuration="1.600843664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:42.599424151 +0000 UTC m=+1138.733414376" watchObservedRunningTime="2026-09-12 00:35:42.600843664 +0000 UTC m=+1138.734833942"
Sep 12 00:35:42.616235 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.616178    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" podStartSLOduration=1.616159286 podStartE2EDuration="1.616159286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:42.613368961 +0000 UTC m=+1138.747359177" watchObservedRunningTime="2026-09-12 00:35:42.616159286 +0000 UTC m=+1138.750149512"
Sep 12 00:35:42.798108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:42.798071    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:35:53.595524 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:53.595483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:35:53.596001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:53.595679    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:35:57.818628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:57.818578    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" podUID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" containerName="main" containerID="cri-o://49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0" gracePeriod=45
Sep 12 00:35:58.075571 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.075499    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:58.216737 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216701    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.216944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.216944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjqm2\" (UniqueName: \"kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.216944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216886    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.216944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.216944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.216921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm\") pod \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\" (UID: \"f349c7b0-7b55-40bd-9925-62a9dfd6af6b\") "
Sep 12 00:35:58.217222 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:58.217222 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home" (OuterVolumeSpecName: "home") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:58.217222 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217182    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache" (OuterVolumeSpecName: "model-cache") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:58.217362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217288    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.217362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217320    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.217362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.217328    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.219107 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.219084    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:58.219192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.219090    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm" (OuterVolumeSpecName: "dshm") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:58.219192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.219119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2" (OuterVolumeSpecName: "kube-api-access-zjqm2") pod "f349c7b0-7b55-40bd-9925-62a9dfd6af6b" (UID: "f349c7b0-7b55-40bd-9925-62a9dfd6af6b"). InnerVolumeSpecName "kube-api-access-zjqm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:58.318262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.318231    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.318262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.318258    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjqm2\" (UniqueName: \"kubernetes.io/projected/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-kube-api-access-zjqm2\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.318262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.318270    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f349c7b0-7b55-40bd-9925-62a9dfd6af6b-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:35:58.679600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.679569    2580 generic.go:356] "Generic (PLEG): container finished" podID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" containerID="49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0" exitCode=0
Sep 12 00:35:58.679756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.679612    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"
Sep 12 00:35:58.679756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.679657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" event={"ID":"f349c7b0-7b55-40bd-9925-62a9dfd6af6b","Type":"ContainerDied","Data":"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"}
Sep 12 00:35:58.679756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.679697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55" event={"ID":"f349c7b0-7b55-40bd-9925-62a9dfd6af6b","Type":"ContainerDied","Data":"df7b091e9a9ee959d59c310a56b8a62a9ed93cc10e9f9cd64f80967eb015be33"}
Sep 12 00:35:58.679756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.679714    2580 scope.go:117] "RemoveContainer" containerID="49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"
Sep 12 00:35:58.688896 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.688877    2580 scope.go:117] "RemoveContainer" containerID="49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"
Sep 12 00:35:58.689152 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:35:58.689135    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0\": container with ID starting with 49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0 not found: ID does not exist" containerID="49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"
Sep 12 00:35:58.689211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.689159    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0"} err="failed to get container status \"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0\": rpc error: code = NotFound desc = could not find container \"49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0\": container with ID starting with 49615c8abb8173e454e5abf55ae03b151da6b8fc8d43ab4d5ae29867fb104bb0 not found: ID does not exist"
Sep 12 00:35:58.695586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.695562    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:35:58.697942 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:35:58.697917    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-4aaf1a/autores-model-4aaf1a-kserve-5659b8cb7-hvf55"]
Sep 12 00:36:00.374185 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:00.374152    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" path="/var/lib/kubelet/pods/f349c7b0-7b55-40bd-9925-62a9dfd6af6b/volumes"
Sep 12 00:36:00.541033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:00.541000    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:36:00.541282 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:00.541258    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy" containerID="cri-o://cde52180ea277012535336143276c7c5230aa348ff185a865369fc124ab18290" gracePeriod=30
Sep 12 00:36:01.301005 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.300968    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:36:01.301583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.301562    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" containerName="main"
Sep 12 00:36:01.301663 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.301587    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" containerName="main"
Sep 12 00:36:01.301736 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.301723    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f349c7b0-7b55-40bd-9925-62a9dfd6af6b" containerName="main"
Sep 12 00:36:01.306576 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.306552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.308718 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.308698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-03e1dc-openshift-default-dockercfg-fkdqc\""
Sep 12 00:36:01.316651 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.316586    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:36:01.449930 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.449893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.449934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.449992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqkj9\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.450306 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.450183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.550900 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.550857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.550913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.550941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.550957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.550979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkj9\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551529 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551751 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.551836 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.551822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.553459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.553439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.553459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.553453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.558592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.558571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkj9\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.558690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.558602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token\") pod \"e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") " pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.620792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.620715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:01.898646 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.898616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:36:01.900659 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:36:01.900629    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d5c6e7_efac_461f_904b_babcfa00cc38.slice/crio-05beb04d092780c306b6dc41ba8a22843f1a5986ca140fa815db22b001fd3257 WatchSource:0}: Error finding container 05beb04d092780c306b6dc41ba8a22843f1a5986ca140fa815db22b001fd3257: Status 404 returned error can't find the container with id 05beb04d092780c306b6dc41ba8a22843f1a5986ca140fa815db22b001fd3257
Sep 12 00:36:01.903339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.903304    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:01.903420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.903401    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:01.903459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:01.903444    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:02.705019 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:02.704966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerStarted","Data":"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"}
Sep 12 00:36:02.705516 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:02.705042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerStarted","Data":"05beb04d092780c306b6dc41ba8a22843f1a5986ca140fa815db22b001fd3257"}
Sep 12 00:36:02.733632 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:02.733552    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podStartSLOduration=1.7335288530000001 podStartE2EDuration="1.733528853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:36:02.728499864 +0000 UTC m=+1158.862490093" watchObservedRunningTime="2026-09-12 00:36:02.733528853 +0000 UTC m=+1158.867519079"
Sep 12 00:36:03.621127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:03.621082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:03.626480 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:03.626453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:03.710745 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:03.710721    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:03.711843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:03.711823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:05.726316 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.726288    2580 generic.go:356] "Generic (PLEG): container finished" podID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerID="cde52180ea277012535336143276c7c5230aa348ff185a865369fc124ab18290" exitCode=0
Sep 12 00:36:05.726720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.726363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerDied","Data":"cde52180ea277012535336143276c7c5230aa348ff185a865369fc124ab18290"}
Sep 12 00:36:05.726720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.726442    2580 scope.go:117] "RemoveContainer" containerID="d3278751c2704a53d4e54b56622399c4228271d128124e5967dacc3447647351"
Sep 12 00:36:05.790839 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.790816    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:36:05.894054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.893955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894086    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894109    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhcjj\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894265    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket\") pod \"a816e271-0eb7-441e-81e7-0e334d5e0a12\" (UID: \"a816e271-0eb7-441e-81e7-0e334d5e0a12\") "
Sep 12 00:36:05.894655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:05.894655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894416    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data" (OuterVolumeSpecName: "istio-data") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:05.894655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894623    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:36:05.894867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894686    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:05.894867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894805    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.894867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894822    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.895006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.894905    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:05.896353 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.896320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:36:05.896597 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.896571    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:05.896597 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.896579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj" (OuterVolumeSpecName: "kube-api-access-fhcjj") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "kube-api-access-fhcjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:05.896713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.896651    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token" (OuterVolumeSpecName: "istio-token") pod "a816e271-0eb7-441e-81e7-0e334d5e0a12" (UID: "a816e271-0eb7-441e-81e7-0e334d5e0a12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:05.995362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995323    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhcjj\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-kube-api-access-fhcjj\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995354    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995363    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995372    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a816e271-0eb7-441e-81e7-0e334d5e0a12-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995385    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995397    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:05.995622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:05.995405    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a816e271-0eb7-441e-81e7-0e334d5e0a12-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:06.595803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.595704    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"]
Sep 12 00:36:06.596121 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.596107    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:06.596176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.596122    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:06.596176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.596134    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:06.596176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.596140    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:06.596275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.596199    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:06.599380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.599363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:06.601384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.601364    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:36:06.616826 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.616753    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"]
Sep 12 00:36:06.703023 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.702983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr\" (UID: \"70a52d62-3a23-4743-9865-cd520ff245d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:06.734069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.734035    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"
Sep 12 00:36:06.734446 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.734065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj" event={"ID":"a816e271-0eb7-441e-81e7-0e334d5e0a12","Type":"ContainerDied","Data":"47a5b835b37ebea7165ec30cf2b4916326e0ef063bcc03fe336e5c391b7e1037"}
Sep 12 00:36:06.734446 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.734133    2580 scope.go:117] "RemoveContainer" containerID="cde52180ea277012535336143276c7c5230aa348ff185a865369fc124ab18290"
Sep 12 00:36:06.750737 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.750702    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:36:06.754234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.754211    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-4aaf1a-openshift-default-75d6b5f594-txrrj"]
Sep 12 00:36:06.803919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.803889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr\" (UID: \"70a52d62-3a23-4743-9865-cd520ff245d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:06.804503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.804484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr\" (UID: \"70a52d62-3a23-4743-9865-cd520ff245d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:06.916407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:06.916371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:07.186851 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:07.186825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"]
Sep 12 00:36:07.188522 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:36:07.188486    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a52d62_3a23_4743_9865_cd520ff245d0.slice/crio-475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1 WatchSource:0}: Error finding container 475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1: Status 404 returned error can't find the container with id 475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1
Sep 12 00:36:07.190572 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:07.190554    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:36:07.741581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:07.741543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr" event={"ID":"70a52d62-3a23-4743-9865-cd520ff245d0","Type":"ContainerStarted","Data":"475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1"}
Sep 12 00:36:08.134103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.134010    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:36:08.134500 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.134485    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" containerName="istio-proxy"
Sep 12 00:36:08.138843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.138818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.141000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.140980    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-03e1dc-serving-cert\""
Sep 12 00:36:08.151289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.151213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:36:08.319189 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.319156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.319367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.319206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.319367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.319305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.319439 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.319388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trc4l\" (UniqueName: \"kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.375201 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.375166    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a816e271-0eb7-441e-81e7-0e334d5e0a12" path="/var/lib/kubelet/pods/a816e271-0eb7-441e-81e7-0e334d5e0a12/volumes"
Sep 12 00:36:08.420800 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.420744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.420944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.420866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trc4l\" (UniqueName: \"kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.420944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.420922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.421025 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.420976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.421131 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:36:08.421114    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-serving-cert: secret "maas-api-e2e-explicit-03e1dc-serving-cert" not found
Sep 12 00:36:08.421198 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:36:08.421187    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls podName:469ba9b0-952e-4334-a0fb-dbec10118ef7 nodeName:}" failed. No retries permitted until 2026-09-12 00:36:08.921166576 +0000 UTC m=+1165.055156786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls") pod "maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7") : secret "maas-api-e2e-explicit-03e1dc-serving-cert" not found
Sep 12 00:36:08.421533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.421496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.423344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.423320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.431271 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.431245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trc4l\" (UniqueName: \"kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.927788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:08.927731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:08.928164 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:36:08.927895    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-serving-cert: secret "maas-api-e2e-explicit-03e1dc-serving-cert" not found
Sep 12 00:36:08.928164 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:36:08.927977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls podName:469ba9b0-952e-4334-a0fb-dbec10118ef7 nodeName:}" failed. No retries permitted until 2026-09-12 00:36:09.927956192 +0000 UTC m=+1166.061946411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls") pod "maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7") : secret "maas-api-e2e-explicit-03e1dc-serving-cert" not found
Sep 12 00:36:09.939187 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:09.939142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:09.941546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:09.941520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:09.953490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:09.953463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:10.233860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:10.233835    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:36:10.235208 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:36:10.235180    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469ba9b0_952e_4334_a0fb_dbec10118ef7.slice/crio-5f10ec7c9386eeefa56976680df2a8426f13e4b9bda464a38ecbc06a72d69138 WatchSource:0}: Error finding container 5f10ec7c9386eeefa56976680df2a8426f13e4b9bda464a38ecbc06a72d69138: Status 404 returned error can't find the container with id 5f10ec7c9386eeefa56976680df2a8426f13e4b9bda464a38ecbc06a72d69138
Sep 12 00:36:10.761819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:10.761780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" event={"ID":"469ba9b0-952e-4334-a0fb-dbec10118ef7","Type":"ContainerStarted","Data":"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"}
Sep 12 00:36:10.762002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:10.761875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" event={"ID":"469ba9b0-952e-4334-a0fb-dbec10118ef7","Type":"ContainerStarted","Data":"5f10ec7c9386eeefa56976680df2a8426f13e4b9bda464a38ecbc06a72d69138"}
Sep 12 00:36:10.762002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:10.761940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:10.780553 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:10.780499    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" podStartSLOduration=2.475427677 podStartE2EDuration="2.78048246s" totalImagesPullingTime="305.054783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:36:10.777451996 +0000 UTC m=+1166.911442220" watchObservedRunningTime="2026-09-12 00:36:10.78048246 +0000 UTC m=+1166.914472685"
Sep 12 00:36:16.775685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:16.775656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:36:21.322885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.322827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:36:21.341787 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.327859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.344175 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.344145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-3c466c\"/\"default-dockercfg-m5sdb\""
Sep 12 00:36:21.345984 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.345948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-3c466c\"/\"kube-root-ca.crt\""
Sep 12 00:36:21.346116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.346070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-3c466c\"/\"openshift-service-ca.crt\""
Sep 12 00:36:21.346268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.346246    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-3c466c\"/\"premium-w0-3c466c-kserve-self-signed-certs\""
Sep 12 00:36:21.349227 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.349203    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:36:21.352622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6z6d\" (UniqueName: \"kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.352706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.352706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.352813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.352856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.352902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.352863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454073 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6z6d\" (UniqueName: \"kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454630 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454694 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.454739 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.454689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.456593 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.456569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.457004 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.456983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.465350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.465329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6z6d\" (UniqueName: \"kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d\") pod \"premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") " pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.654386 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.654271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:21.950554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:21.950515    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:36:21.954699 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:36:21.954666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0f8571f_4feb_4d3b_a175_a16da7c63434.slice/crio-c9da24c985e053f856c5de797e4e366d27830fa19a573d0545b0008ab96f31f2 WatchSource:0}: Error finding container c9da24c985e053f856c5de797e4e366d27830fa19a573d0545b0008ab96f31f2: Status 404 returned error can't find the container with id c9da24c985e053f856c5de797e4e366d27830fa19a573d0545b0008ab96f31f2
Sep 12 00:36:22.861127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:22.861010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" event={"ID":"c0f8571f-4feb-4d3b-a175-a16da7c63434","Type":"ContainerStarted","Data":"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"}
Sep 12 00:36:22.861127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:22.861069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" event={"ID":"c0f8571f-4feb-4d3b-a175-a16da7c63434","Type":"ContainerStarted","Data":"c9da24c985e053f856c5de797e4e366d27830fa19a573d0545b0008ab96f31f2"}
Sep 12 00:36:22.877946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:22.877836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" podStartSLOduration=1.877819849 podStartE2EDuration="1.877819849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:36:22.87645211 +0000 UTC m=+1179.010442371" watchObservedRunningTime="2026-09-12 00:36:22.877819849 +0000 UTC m=+1179.011810074"
Sep 12 00:36:24.874062 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:24.873963    2580 generic.go:356] "Generic (PLEG): container finished" podID="70a52d62-3a23-4743-9865-cd520ff245d0" containerID="0213dad798844d17702efe3a9dc114b0afd19475a7aed250d8fadc45ab271f92" exitCode=0
Sep 12 00:36:24.874550 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:24.874057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr" event={"ID":"70a52d62-3a23-4743-9865-cd520ff245d0","Type":"ContainerDied","Data":"0213dad798844d17702efe3a9dc114b0afd19475a7aed250d8fadc45ab271f92"}
Sep 12 00:36:26.015418 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.015391    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:26.107780 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.107739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca\") pod \"70a52d62-3a23-4743-9865-cd520ff245d0\" (UID: \"70a52d62-3a23-4743-9865-cd520ff245d0\") "
Sep 12 00:36:26.108122 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.108095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70a52d62-3a23-4743-9865-cd520ff245d0" (UID: "70a52d62-3a23-4743-9865-cd520ff245d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:36:26.209323 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.209290    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a52d62-3a23-4743-9865-cd520ff245d0-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:26.886562 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.886535    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"
Sep 12 00:36:26.886756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.886580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr" event={"ID":"70a52d62-3a23-4743-9865-cd520ff245d0","Type":"ContainerDied","Data":"475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1"}
Sep 12 00:36:26.886756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:26.886639    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="475949b961d71c12675fe971528f5b07d42ac276da0d7e503339a5dc66c590a1"
Sep 12 00:36:27.894247 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.894150    2580 generic.go:356] "Generic (PLEG): container finished" podID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerID="75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5" exitCode=0
Sep 12 00:36:27.894247 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.894218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerDied","Data":"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"}
Sep 12 00:36:27.894672 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.894656    2580 scope.go:117] "RemoveContainer" containerID="75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"
Sep 12 00:36:27.895856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.895820    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:27.895960 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.895895    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:27.895960 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:27.895926    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:36:28.748157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.748119    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:36:28.748638 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.748590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" podUID="72dcc666-3cff-4ead-8dba-44bd11408651" containerName="payload-processing" containerID="cri-o://1a987a5e28dae374daaad5b7f91b6106f29d52257bcd15ba283b0bd9fd4c41c7" gracePeriod=30
Sep 12 00:36:28.754902 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.754796    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:36:28.755060 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.755033    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" podUID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" containerName="payload-pre-processing" containerID="cri-o://53a08f721e48530a8a1086cd1345c83d5084b524bde0921ee480f171467d2dba" gracePeriod=30
Sep 12 00:36:28.904648 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.904602    2580 generic.go:356] "Generic (PLEG): container finished" podID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" containerID="53a08f721e48530a8a1086cd1345c83d5084b524bde0921ee480f171467d2dba" exitCode=0
Sep 12 00:36:28.905159 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.904694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" event={"ID":"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9","Type":"ContainerDied","Data":"53a08f721e48530a8a1086cd1345c83d5084b524bde0921ee480f171467d2dba"}
Sep 12 00:36:28.906001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.905970    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"]
Sep 12 00:36:28.907224 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.907196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerStarted","Data":"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"}
Sep 12 00:36:28.908972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.908934    2580 generic.go:356] "Generic (PLEG): container finished" podID="72dcc666-3cff-4ead-8dba-44bd11408651" containerID="1a987a5e28dae374daaad5b7f91b6106f29d52257bcd15ba283b0bd9fd4c41c7" exitCode=0
Sep 12 00:36:28.909090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.908979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" event={"ID":"72dcc666-3cff-4ead-8dba-44bd11408651","Type":"ContainerDied","Data":"1a987a5e28dae374daaad5b7f91b6106f29d52257bcd15ba283b0bd9fd4c41c7"}
Sep 12 00:36:28.909712 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:28.909687    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9c414efe-kqtzr"]
Sep 12 00:36:29.075539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.074600    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:36:29.109742 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.109714    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:36:29.149278 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.149240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgdzs\" (UniqueName: \"kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs\") pod \"72dcc666-3cff-4ead-8dba-44bd11408651\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") "
Sep 12 00:36:29.149569 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.149325    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume\") pod \"72dcc666-3cff-4ead-8dba-44bd11408651\" (UID: \"72dcc666-3cff-4ead-8dba-44bd11408651\") "
Sep 12 00:36:29.150531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.150490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72dcc666-3cff-4ead-8dba-44bd11408651" (UID: "72dcc666-3cff-4ead-8dba-44bd11408651"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:36:29.153347 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.153269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs" (OuterVolumeSpecName: "kube-api-access-dgdzs") pod "72dcc666-3cff-4ead-8dba-44bd11408651" (UID: "72dcc666-3cff-4ead-8dba-44bd11408651"). InnerVolumeSpecName "kube-api-access-dgdzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:29.251047 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.251015    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume\") pod \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") "
Sep 12 00:36:29.251248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.251065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlhz6\" (UniqueName: \"kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6\") pod \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\" (UID: \"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9\") "
Sep 12 00:36:29.251351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.251337    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgdzs\" (UniqueName: \"kubernetes.io/projected/72dcc666-3cff-4ead-8dba-44bd11408651-kube-api-access-dgdzs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:29.251424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.251352    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dcc666-3cff-4ead-8dba-44bd11408651-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:29.251477 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.251432    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" (UID: "05a2eee8-04b2-4f74-94fb-08b6b3d5cca9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:36:29.253141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.253120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6" (OuterVolumeSpecName: "kube-api-access-mlhz6") pod "05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" (UID: "05a2eee8-04b2-4f74-94fb-08b6b3d5cca9"). InnerVolumeSpecName "kube-api-access-mlhz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:29.352962 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.352875    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:29.352962 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.352908    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlhz6\" (UniqueName: \"kubernetes.io/projected/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9-kube-api-access-mlhz6\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:36:29.622979 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.621300    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:29.622979 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.622579    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 12 00:36:29.622979 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.622635    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 12 00:36:29.918051 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.917951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb" event={"ID":"05a2eee8-04b2-4f74-94fb-08b6b3d5cca9","Type":"ContainerDied","Data":"fdd7fd0f97cf13de8e550f5b07548917fcf9150aedfe8ca9811bfaecb1c50c99"}
Sep 12 00:36:29.918609 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.918588    2580 scope.go:117] "RemoveContainer" containerID="53a08f721e48530a8a1086cd1345c83d5084b524bde0921ee480f171467d2dba"
Sep 12 00:36:29.918911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.918892    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"
Sep 12 00:36:29.939810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.937481    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"
Sep 12 00:36:29.947104 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.947075    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6" event={"ID":"72dcc666-3cff-4ead-8dba-44bd11408651","Type":"ContainerDied","Data":"93501bec72e48e967a46604c343233fb04c9cdc6b280e063d7bbfe9a1dc4b46c"}
Sep 12 00:36:29.954641 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.954618    2580 scope.go:117] "RemoveContainer" containerID="1a987a5e28dae374daaad5b7f91b6106f29d52257bcd15ba283b0bd9fd4c41c7"
Sep 12 00:36:29.961191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.961163    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:36:29.966218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.966183    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9c414efe-7876dfb886-4mflb"]
Sep 12 00:36:29.979973 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.979942    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:36:29.984947 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:29.984912    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9c414efe-74dbd86f6f-gg4t6"]
Sep 12 00:36:30.374599 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:30.374566    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" path="/var/lib/kubelet/pods/05a2eee8-04b2-4f74-94fb-08b6b3d5cca9/volumes"
Sep 12 00:36:30.374955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:30.374941    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70a52d62-3a23-4743-9865-cd520ff245d0" path="/var/lib/kubelet/pods/70a52d62-3a23-4743-9865-cd520ff245d0/volumes"
Sep 12 00:36:30.375260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:30.375248    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72dcc666-3cff-4ead-8dba-44bd11408651" path="/var/lib/kubelet/pods/72dcc666-3cff-4ead-8dba-44bd11408651/volumes"
Sep 12 00:36:30.621362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:30.621325    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 12 00:36:30.621666 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:30.621638    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 12 00:36:31.621846 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.621802    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 12 00:36:31.622454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.622415    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 12 00:36:31.654559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.654525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:31.654725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.654604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:31.666911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.666884    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:31.963049 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:31.963003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:36:32.622194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:32.622155    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 12 00:36:32.622852 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:32.622811    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 12 00:36:33.621460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:33.621418    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 12 00:36:33.621818 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:33.621785    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 12 00:36:33.711041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:33.711002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:34.626510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:34.626482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:34.627379 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:34.627357    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:36:38.819409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819377    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819736    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" containerName="payload-pre-processing"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" containerName="payload-pre-processing"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819755    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a52d62-3a23-4743-9865-cd520ff245d0" containerName="revoke-keys"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819761    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a52d62-3a23-4743-9865-cd520ff245d0" containerName="revoke-keys"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819813    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dcc666-3cff-4ead-8dba-44bd11408651" containerName="payload-processing"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819820    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dcc666-3cff-4ead-8dba-44bd11408651" containerName="payload-processing"
Sep 12 00:36:38.819882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819885    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a52d62-3a23-4743-9865-cd520ff245d0" containerName="revoke-keys"
Sep 12 00:36:38.820262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819893    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dcc666-3cff-4ead-8dba-44bd11408651" containerName="payload-processing"
Sep 12 00:36:38.820262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.819901    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a2eee8-04b2-4f74-94fb-08b6b3d5cca9" containerName="payload-pre-processing"
Sep 12 00:36:38.827173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.827151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.830749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.830480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"default-dockercfg-sh6fv\""
Sep 12 00:36:38.830749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.830541    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"openshift-service-ca.crt\""
Sep 12 00:36:38.830749 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.830625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"premium-w4-139288-kserve-self-signed-certs\""
Sep 12 00:36:38.831310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.831285    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"kube-root-ca.crt\""
Sep 12 00:36:38.834283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.834262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:36:38.955980 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.955945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.956158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.955994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.956158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.956106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.956158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.956152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.956328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.956241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:38.956328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:38.956278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhg49\" (UniqueName: \"kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.056840 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhg49\" (UniqueName: \"kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057041 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.056967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057305 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.057256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057371 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.057300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.057371 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.057347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.059260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.059230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.059554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.059535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.064855 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.064833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhg49\" (UniqueName: \"kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49\") pod \"premium-w4-139288-kserve-768c498bf7-czlnz\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") " pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.137517 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.137426    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:39.409831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.409809    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:36:39.411517 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:36:39.411489    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75b81ef8_fe97_48ef_b9c3_0e808f7d3809.slice/crio-31784d7ac4d8706ee7972823a1016c7ad1847daf0d5964d51883b129a7ee4425 WatchSource:0}: Error finding container 31784d7ac4d8706ee7972823a1016c7ad1847daf0d5964d51883b129a7ee4425: Status 404 returned error can't find the container with id 31784d7ac4d8706ee7972823a1016c7ad1847daf0d5964d51883b129a7ee4425
Sep 12 00:36:39.997813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.997775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" event={"ID":"75b81ef8-fe97-48ef-b9c3-0e808f7d3809","Type":"ContainerStarted","Data":"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"}
Sep 12 00:36:39.997813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:39.997819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" event={"ID":"75b81ef8-fe97-48ef-b9c3-0e808f7d3809","Type":"ContainerStarted","Data":"31784d7ac4d8706ee7972823a1016c7ad1847daf0d5964d51883b129a7ee4425"}
Sep 12 00:36:40.013700 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:40.013650    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" podStartSLOduration=2.013635327 podStartE2EDuration="2.013635327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:36:40.012052246 +0000 UTC m=+1196.146042462" watchObservedRunningTime="2026-09-12 00:36:40.013635327 +0000 UTC m=+1196.147625553"
Sep 12 00:36:49.138211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:49.138172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:49.138606 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:49.138482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:49.151275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:49.151254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:50.064378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:50.064346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:36:59.819451 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.819416    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:36:59.824496 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.824470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.827634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.827611    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-39d42b\"/\"openshift-service-ca.crt\""
Sep 12 00:36:59.827779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.827641    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-39d42b\"/\"kube-root-ca.crt\""
Sep 12 00:36:59.827779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.827612    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-39d42b\"/\"default-dockercfg-cm52j\""
Sep 12 00:36:59.827779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.827612    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-39d42b\"/\"test-model-39d42b-kserve-self-signed-certs\""
Sep 12 00:36:59.832954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.832927    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:36:59.869636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztg4d\" (UniqueName: \"kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.869636 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.869889 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.869889 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.869889 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.869889 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.869800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.970972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.970937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztg4d\" (UniqueName: \"kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.970972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.970978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971241 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971584 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.971584 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.971556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.973429 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.973410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.973686 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.973669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:36:59.978637 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:36:59.978614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztg4d\" (UniqueName: \"kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d\") pod \"test-model-39d42b-kserve-f6f8bdfdf-mqzs7\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") " pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:00.136410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:00.136313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:00.406119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:00.406050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:37:00.409743 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:37:00.409718    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55b35d18_1403_49bb_ba94_e101d5e60333.slice/crio-a19b08c6b82ef77bcac739007a666ecb8577157cf46e8a6a773e98fb96146b69 WatchSource:0}: Error finding container a19b08c6b82ef77bcac739007a666ecb8577157cf46e8a6a773e98fb96146b69: Status 404 returned error can't find the container with id a19b08c6b82ef77bcac739007a666ecb8577157cf46e8a6a773e98fb96146b69
Sep 12 00:37:01.117990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:01.117949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" event={"ID":"55b35d18-1403-49bb-ba94-e101d5e60333","Type":"ContainerStarted","Data":"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"}
Sep 12 00:37:01.118387 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:01.118000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" event={"ID":"55b35d18-1403-49bb-ba94-e101d5e60333","Type":"ContainerStarted","Data":"a19b08c6b82ef77bcac739007a666ecb8577157cf46e8a6a773e98fb96146b69"}
Sep 12 00:37:01.134903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:01.134854    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" podStartSLOduration=2.13483874 podStartE2EDuration="2.13483874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:37:01.131717046 +0000 UTC m=+1217.265707271" watchObservedRunningTime="2026-09-12 00:37:01.13483874 +0000 UTC m=+1217.268828991"
Sep 12 00:37:10.136665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:10.136626    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:10.137117 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:10.136684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:10.149676 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:10.149647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:10.183914 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:10.183889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:37:39.775153 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:39.775060    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:37:39.775737 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:39.775276    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" podUID="469ba9b0-952e-4334-a0fb-dbec10118ef7" containerName="maas-api" containerID="cri-o://fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7" gracePeriod=30
Sep 12 00:37:40.036487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.036424    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:37:40.174224 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.174171    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trc4l\" (UniqueName: \"kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l\") pod \"469ba9b0-952e-4334-a0fb-dbec10118ef7\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") "
Sep 12 00:37:40.174465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.174264    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") pod \"469ba9b0-952e-4334-a0fb-dbec10118ef7\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") "
Sep 12 00:37:40.174465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.174352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs\") pod \"469ba9b0-952e-4334-a0fb-dbec10118ef7\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") "
Sep 12 00:37:40.174465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.174426    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca\") pod \"469ba9b0-952e-4334-a0fb-dbec10118ef7\" (UID: \"469ba9b0-952e-4334-a0fb-dbec10118ef7\") "
Sep 12 00:37:40.175123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.175076    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "469ba9b0-952e-4334-a0fb-dbec10118ef7" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:37:40.178876 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.178746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l" (OuterVolumeSpecName: "kube-api-access-trc4l") pod "469ba9b0-952e-4334-a0fb-dbec10118ef7" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7"). InnerVolumeSpecName "kube-api-access-trc4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:37:40.178876 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.178850    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "469ba9b0-952e-4334-a0fb-dbec10118ef7" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:37:40.179059 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.178904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "469ba9b0-952e-4334-a0fb-dbec10118ef7" (UID: "469ba9b0-952e-4334-a0fb-dbec10118ef7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:37:40.275985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.275952    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trc4l\" (UniqueName: \"kubernetes.io/projected/469ba9b0-952e-4334-a0fb-dbec10118ef7-kube-api-access-trc4l\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.275985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.275980    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-maas-api-tls\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.275985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.275989    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469ba9b0-952e-4334-a0fb-dbec10118ef7-metrics-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.276211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.275998    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469ba9b0-952e-4334-a0fb-dbec10118ef7-openshift-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.346580 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.346487    2580 generic.go:356] "Generic (PLEG): container finished" podID="469ba9b0-952e-4334-a0fb-dbec10118ef7" containerID="fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7" exitCode=0
Sep 12 00:37:40.346580 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.346527    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"
Sep 12 00:37:40.346780 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.346577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" event={"ID":"469ba9b0-952e-4334-a0fb-dbec10118ef7","Type":"ContainerDied","Data":"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"}
Sep 12 00:37:40.346780 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.346627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt" event={"ID":"469ba9b0-952e-4334-a0fb-dbec10118ef7","Type":"ContainerDied","Data":"5f10ec7c9386eeefa56976680df2a8426f13e4b9bda464a38ecbc06a72d69138"}
Sep 12 00:37:40.346780 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.346648    2580 scope.go:117] "RemoveContainer" containerID="fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"
Sep 12 00:37:40.358803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.358777    2580 scope.go:117] "RemoveContainer" containerID="fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"
Sep 12 00:37:40.359113 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:37:40.359095    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7\": container with ID starting with fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7 not found: ID does not exist" containerID="fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"
Sep 12 00:37:40.359188 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.359125    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7"} err="failed to get container status \"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7\": rpc error: code = NotFound desc = could not find container \"fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7\": container with ID starting with fcf553e34944109db6f2854aac69e99d7d5b57f02e77dabef0c7d5f5ccddf5e7 not found: ID does not exist"
Sep 12 00:37:40.375236 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.375207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:37:40.376727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:40.376706    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-03e1dc-984b5c4f7-w9hpt"]
Sep 12 00:37:42.374032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:42.373994    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469ba9b0-952e-4334-a0fb-dbec10118ef7" path="/var/lib/kubelet/pods/469ba9b0-952e-4334-a0fb-dbec10118ef7/volumes"
Sep 12 00:37:46.438572 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:46.438540    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:37:46.439015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:46.438748    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" containerID="cri-o://e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807" gracePeriod=30
Sep 12 00:37:49.629307 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:49.629266    2580 patch_prober.go:28] interesting pod/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:37:49.629695 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:49.629343    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:37:51.693302 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.693275    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:37:51.787549 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787463    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787549 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787511    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787549 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqkj9\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787589    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787792    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy\") pod \"82d5c6e7-efac-461f-904b-babcfa00cc38\" (UID: \"82d5c6e7-efac-461f-904b-babcfa00cc38\") "
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787828    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:37:51.787866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.787839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:37:51.788271 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:37:51.788338 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data" (OuterVolumeSpecName: "istio-data") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:37:51.788390 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788332    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.788390 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788353    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.788390 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788367    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.788527 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.788484    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:37:51.789948 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.789923    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9" (OuterVolumeSpecName: "kube-api-access-tqkj9") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "kube-api-access-tqkj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:37:51.790220 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.790200    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:37:51.790391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.790371    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:37:51.790457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.790373    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token" (OuterVolumeSpecName: "istio-token") pod "82d5c6e7-efac-461f-904b-babcfa00cc38" (UID: "82d5c6e7-efac-461f-904b-babcfa00cc38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:37:51.889310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889275    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqkj9\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-kube-api-access-tqkj9\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.889310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889306    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.889310 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889316    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.889554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889326    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82d5c6e7-efac-461f-904b-babcfa00cc38-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.889554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889334    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:51.889554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:51.889342    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82d5c6e7-efac-461f-904b-babcfa00cc38-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:37:52.420358 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.420325    2580 generic.go:356] "Generic (PLEG): container finished" podID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerID="e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807" exitCode=0
Sep 12 00:37:52.420523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.420387    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"
Sep 12 00:37:52.420523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.420415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerDied","Data":"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"}
Sep 12 00:37:52.420523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.420467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr" event={"ID":"82d5c6e7-efac-461f-904b-babcfa00cc38","Type":"ContainerDied","Data":"05beb04d092780c306b6dc41ba8a22843f1a5986ca140fa815db22b001fd3257"}
Sep 12 00:37:52.420523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.420493    2580 scope.go:117] "RemoveContainer" containerID="e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"
Sep 12 00:37:52.429816 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.429791    2580 scope.go:117] "RemoveContainer" containerID="75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"
Sep 12 00:37:52.438535 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.438519    2580 scope.go:117] "RemoveContainer" containerID="e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"
Sep 12 00:37:52.438874 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:37:52.438851    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807\": container with ID starting with e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807 not found: ID does not exist" containerID="e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"
Sep 12 00:37:52.438955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.438886    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807"} err="failed to get container status \"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807\": rpc error: code = NotFound desc = could not find container \"e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807\": container with ID starting with e0d97d65a696a9c9ae65b88e7eab0503d303285f84862874f3a9cc191a5dd807 not found: ID does not exist"
Sep 12 00:37:52.438955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.438905    2580 scope.go:117] "RemoveContainer" containerID="75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"
Sep 12 00:37:52.439198 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:37:52.439174    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5\": container with ID starting with 75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5 not found: ID does not exist" containerID="75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"
Sep 12 00:37:52.439238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.439207    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5"} err="failed to get container status \"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5\": rpc error: code = NotFound desc = could not find container \"75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5\": container with ID starting with 75b424ac3ed6bc44a0f48869543ff110a8a98d66df1e6817efca8bdc9f08dee5 not found: ID does not exist"
Sep 12 00:37:52.439904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.439886    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:37:52.443745 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:52.443721    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-03e1dc-openshift-default-86cc87b4b4-298lr"]
Sep 12 00:37:54.375198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:37:54.375167    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" path="/var/lib/kubelet/pods/82d5c6e7-efac-461f-904b-babcfa00cc38/volumes"
Sep 12 00:38:04.824320 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.824283    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:38:04.824930 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.824904    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:04.824930 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.824927    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:04.825093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.824938    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469ba9b0-952e-4334-a0fb-dbec10118ef7" containerName="maas-api"
Sep 12 00:38:04.825093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.824947    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="469ba9b0-952e-4334-a0fb-dbec10118ef7" containerName="maas-api"
Sep 12 00:38:04.825093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.825057    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:04.825093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.825067    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="469ba9b0-952e-4334-a0fb-dbec10118ef7" containerName="maas-api"
Sep 12 00:38:04.825093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.825078    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:04.837520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.837489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.839905 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.839877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d370bb1a-openshift-default-dockercfg-8bdr8\""
Sep 12 00:38:04.840543 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.840509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:38:04.910893 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.910844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqkcx\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.910905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.910960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.910993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.911012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.911034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911083 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.911050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911288 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.911112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:04.911288 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:04.911194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqkcx\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.012837 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.012959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013270 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.013231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.013334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013522 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.013471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013594 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.013543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.013667 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.013650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.015156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.015131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.015264 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.015248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.020457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.020435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.020586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.020564    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqkcx\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx\") pod \"e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") " pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.149669 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.149583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:05.425357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.425323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:38:05.428287 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:38:05.428256    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d234a96_9d42_44ed_97c8_8fb40b46bd9b.slice/crio-442c04e2e20a0fc672df21b851e8845189c0ca70bfb4856e173a46fd9adfaef7 WatchSource:0}: Error finding container 442c04e2e20a0fc672df21b851e8845189c0ca70bfb4856e173a46fd9adfaef7: Status 404 returned error can't find the container with id 442c04e2e20a0fc672df21b851e8845189c0ca70bfb4856e173a46fd9adfaef7
Sep 12 00:38:05.430541 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.430506    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:05.430613 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.430573    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:05.430613 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.430604    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:05.504760 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.504729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerStarted","Data":"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"}
Sep 12 00:38:05.504919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.504802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerStarted","Data":"442c04e2e20a0fc672df21b851e8845189c0ca70bfb4856e173a46fd9adfaef7"}
Sep 12 00:38:05.542675 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:05.542621    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podStartSLOduration=1.542603561 podStartE2EDuration="1.542603561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:05.539474391 +0000 UTC m=+1281.673464628" watchObservedRunningTime="2026-09-12 00:38:05.542603561 +0000 UTC m=+1281.676593786"
Sep 12 00:38:06.150290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:06.150251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:06.155719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:06.155691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:06.510726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:06.510691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:06.511633 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:06.511612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:34.522927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.522892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:38:34.523424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.523280    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:34.523424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.523294    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d5c6e7-efac-461f-904b-babcfa00cc38" containerName="istio-proxy"
Sep 12 00:38:34.526589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.526564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.529080 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.529054    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-3c466c\"/\"distinct-w0-3c466c-kserve-self-signed-certs\""
Sep 12 00:38:34.537754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.537729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:38:34.711535 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdcmf\" (UniqueName: \"kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.711704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.711704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.711847 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.711847 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.711847 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.711782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdcmf\" (UniqueName: \"kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.812887 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.812668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.813054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.813014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.813116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.813064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.813172 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.813129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.814964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.814936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.815275 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.815254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.821145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.821122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdcmf\" (UniqueName: \"kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf\") pod \"distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") " pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:34.837590 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:34.837564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:35.125468 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:35.125435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:38:35.127024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:38:35.126993    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24119ca5_6c0c_4aed_a0af_9a364149b5de.slice/crio-907c1b575b67a84c62a0246c2b34f74b84ed29f8bddd5ba6468989e46107e82b WatchSource:0}: Error finding container 907c1b575b67a84c62a0246c2b34f74b84ed29f8bddd5ba6468989e46107e82b: Status 404 returned error can't find the container with id 907c1b575b67a84c62a0246c2b34f74b84ed29f8bddd5ba6468989e46107e82b
Sep 12 00:38:35.676617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:35.676574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" event={"ID":"24119ca5-6c0c-4aed-a0af-9a364149b5de","Type":"ContainerStarted","Data":"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"}
Sep 12 00:38:35.676617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:35.676625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" event={"ID":"24119ca5-6c0c-4aed-a0af-9a364149b5de","Type":"ContainerStarted","Data":"907c1b575b67a84c62a0246c2b34f74b84ed29f8bddd5ba6468989e46107e82b"}
Sep 12 00:38:37.551934 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.551891    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 12 00:38:37.552339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.551948    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 12 00:38:37.689509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.689469    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerID="00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4" exitCode=0
Sep 12 00:38:37.689649 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.689536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerDied","Data":"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"}
Sep 12 00:38:37.690051 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.690035    2580 scope.go:117] "RemoveContainer" containerID="00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"
Sep 12 00:38:37.691270 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.691246    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:37.691328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.691309    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:37.691362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.691338    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:38:37.717548 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:37.717491    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" podStartSLOduration=3.7174622189999997 podStartE2EDuration="3.717462219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:35.69289107 +0000 UTC m=+1311.826881295" watchObservedRunningTime="2026-09-12 00:38:37.717462219 +0000 UTC m=+1313.851452445"
Sep 12 00:38:38.697319 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:38.697283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerStarted","Data":"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"}
Sep 12 00:38:39.150547 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.150443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:39.150709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.150615    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:38:39.150709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.150655    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 12 00:38:39.742029 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.741991    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"]
Sep 12 00:38:39.745175 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.745156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:39.747259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.747235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:38:39.753856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.753278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"]
Sep 12 00:38:39.763078 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.763054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk\" (UID: \"6fc06867-6ad4-4b4a-b667-16db00a1fe70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:39.863877 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.863837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk\" (UID: \"6fc06867-6ad4-4b4a-b667-16db00a1fe70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:39.864509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:39.864484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk\" (UID: \"6fc06867-6ad4-4b4a-b667-16db00a1fe70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:40.057556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.057455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:40.150388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.149929    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:38:40.150388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.149992    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 12 00:38:40.341532 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:38:40.341502    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc06867_6ad4_4b4a_b667_16db00a1fe70.slice/crio-1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a WatchSource:0}: Error finding container 1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a: Status 404 returned error can't find the container with id 1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a
Sep 12 00:38:40.342993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.342972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"]
Sep 12 00:38:40.710255 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.710217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk" event={"ID":"6fc06867-6ad4-4b4a-b667-16db00a1fe70","Type":"ContainerStarted","Data":"b5868752e57bf25aa5105fa0c464a7a05b6709e0085c2f428c0699fb6b773e41"}
Sep 12 00:38:40.710431 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.710280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk" event={"ID":"6fc06867-6ad4-4b4a-b667-16db00a1fe70","Type":"ContainerStarted","Data":"1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a"}
Sep 12 00:38:40.727324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:40.727261    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk" podStartSLOduration=1.727239945 podStartE2EDuration="1.727239945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:40.723924936 +0000 UTC m=+1316.857915154" watchObservedRunningTime="2026-09-12 00:38:40.727239945 +0000 UTC m=+1316.861230170"
Sep 12 00:38:41.150732 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:41.150691    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:38:41.151180 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:41.150753    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 12 00:38:41.719608 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:41.719560    2580 generic.go:356] "Generic (PLEG): container finished" podID="6fc06867-6ad4-4b4a-b667-16db00a1fe70" containerID="b5868752e57bf25aa5105fa0c464a7a05b6709e0085c2f428c0699fb6b773e41" exitCode=0
Sep 12 00:38:41.719838 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:41.719640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk" event={"ID":"6fc06867-6ad4-4b4a-b667-16db00a1fe70","Type":"ContainerDied","Data":"b5868752e57bf25aa5105fa0c464a7a05b6709e0085c2f428c0699fb6b773e41"}
Sep 12 00:38:42.151114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:42.151025    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:38:42.151114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:42.151093    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 12 00:38:42.900938 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:42.900914    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:42.997462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:42.997378    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca\") pod \"6fc06867-6ad4-4b4a-b667-16db00a1fe70\" (UID: \"6fc06867-6ad4-4b4a-b667-16db00a1fe70\") "
Sep 12 00:38:42.997750 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:42.997724    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fc06867-6ad4-4b4a-b667-16db00a1fe70" (UID: "6fc06867-6ad4-4b4a-b667-16db00a1fe70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:38:43.098680 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.098648    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc06867-6ad4-4b4a-b667-16db00a1fe70-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:38:43.150405 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.150369    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:38:43.150592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.150430    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 12 00:38:43.735744 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.735717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"
Sep 12 00:38:43.735744 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.735737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk" event={"ID":"6fc06867-6ad4-4b4a-b667-16db00a1fe70","Type":"ContainerDied","Data":"1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a"}
Sep 12 00:38:43.735744 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:43.735804    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ae59e96dfc27b780b56359010a0f4cc89f39fe2c16adad6136179d01619186a"
Sep 12 00:38:44.153946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.153864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:44.154160 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.154140    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:44.154911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.154892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:38:44.838634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.838548    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:44.838634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.838591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:44.851345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:44.851321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:45.758126 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:45.758093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:38:50.716392 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:50.716287    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"]
Sep 12 00:38:50.718882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:50.718858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d370bb1a-gctbk"]
Sep 12 00:38:52.374047 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:52.374012    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fc06867-6ad4-4b4a-b667-16db00a1fe70" path="/var/lib/kubelet/pods/6fc06867-6ad4-4b4a-b667-16db00a1fe70/volumes"
Sep 12 00:38:56.922461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:56.922416    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:38:56.922953 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:56.922684    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" containerID="cri-o://f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8" gracePeriod=30
Sep 12 00:38:59.156418 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:59.156377    2580 patch_prober.go:28] interesting pod/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:38:59.156899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:38:59.156454    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:39:02.181628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.181600    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:39:02.296095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296062    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296205    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296270    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:02.296490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:02.296490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296490 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296479    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:39:02.296759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296488    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqkcx\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx\") pod \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\" (UID: \"2d234a96-9d42-44ed-97c8-8fb40b46bd9b\") "
Sep 12 00:39:02.296759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:02.296759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.296730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:02.297045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.297021    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.297112 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.297049    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.297112 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.297064    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.297112 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.297078    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.297112 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.297090    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.298311 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.298286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:39:02.298481 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.298461    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx" (OuterVolumeSpecName: "kube-api-access-fqkcx") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "kube-api-access-fqkcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:02.298559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.298540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:02.298622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.298593    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d234a96-9d42-44ed-97c8-8fb40b46bd9b" (UID: "2d234a96-9d42-44ed-97c8-8fb40b46bd9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:02.397618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.397575    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqkcx\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-kube-api-access-fqkcx\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.397618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.397610    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.397618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.397623    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.397875 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.397635    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d234a96-9d42-44ed-97c8-8fb40b46bd9b-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:02.840830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.840794    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerID="f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8" exitCode=0
Sep 12 00:39:02.841000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.840866    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"
Sep 12 00:39:02.841000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.840883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerDied","Data":"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"}
Sep 12 00:39:02.841000 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.840979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b" event={"ID":"2d234a96-9d42-44ed-97c8-8fb40b46bd9b","Type":"ContainerDied","Data":"442c04e2e20a0fc672df21b851e8845189c0ca70bfb4856e173a46fd9adfaef7"}
Sep 12 00:39:02.841114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.841011    2580 scope.go:117] "RemoveContainer" containerID="f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"
Sep 12 00:39:02.850270 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.850249    2580 scope.go:117] "RemoveContainer" containerID="00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"
Sep 12 00:39:02.859151 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.859120    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:39:02.862106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.862045    2580 scope.go:117] "RemoveContainer" containerID="f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"
Sep 12 00:39:02.862520 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:39:02.862494    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8\": container with ID starting with f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8 not found: ID does not exist" containerID="f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"
Sep 12 00:39:02.862600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.862535    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8"} err="failed to get container status \"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8\": rpc error: code = NotFound desc = could not find container \"f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8\": container with ID starting with f8e7540b82a386afe6ccfcf1d384861e9f63368345b68494ffcf885cb1884ee8 not found: ID does not exist"
Sep 12 00:39:02.862600 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.862560    2580 scope.go:117] "RemoveContainer" containerID="00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"
Sep 12 00:39:02.862955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.862934    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d370bb1a-openshift-default-6959cdb6c6-vz74b"]
Sep 12 00:39:02.863226 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:39:02.863203    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4\": container with ID starting with 00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4 not found: ID does not exist" containerID="00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"
Sep 12 00:39:02.863277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:02.863235    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4"} err="failed to get container status \"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4\": rpc error: code = NotFound desc = could not find container \"00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4\": container with ID starting with 00f4ab3933b362f866aa91e8f6a9091fdfef0cc523de97a055b76321cce280f4 not found: ID does not exist"
Sep 12 00:39:04.374440 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:04.374402    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" path="/var/lib/kubelet/pods/2d234a96-9d42-44ed-97c8-8fb40b46bd9b/volumes"
Sep 12 00:39:11.110841 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.110744    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111118    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc06867-6ad4-4b4a-b667-16db00a1fe70" containerName="revoke-keys"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111131    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc06867-6ad4-4b4a-b667-16db00a1fe70" containerName="revoke-keys"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111145    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111150    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111162    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111167    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111229    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111239    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d234a96-9d42-44ed-97c8-8fb40b46bd9b" containerName="istio-proxy"
Sep 12 00:39:11.111286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.111244    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fc06867-6ad4-4b4a-b667-16db00a1fe70" containerName="revoke-keys"
Sep 12 00:39:11.115677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.115659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.117867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.117843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-0a0781\"/\"openshift-service-ca.crt\""
Sep 12 00:39:11.118604 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.118584    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-0a0781\"/\"kube-root-ca.crt\""
Sep 12 00:39:11.118691 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.118668    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-0a0781\"/\"noten-model-0a0781-kserve-self-signed-certs\""
Sep 12 00:39:11.118760 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.118677    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-0a0781\"/\"default-dockercfg-wk6mx\""
Sep 12 00:39:11.124137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.124112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:39:11.185449 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.185626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jl4k\" (UniqueName: \"kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.185626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.185626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.185755 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.185755 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.185743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.286761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.286955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jl4k\" (UniqueName: \"kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.286955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.286955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.286955 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.287153 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.286961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.287261 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.287239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.287330 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.287282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.287330 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.287301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.289232 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.289201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.289602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.289582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.295052 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.295032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jl4k\" (UniqueName: \"kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k\") pod \"noten-model-0a0781-kserve-6c6bdd9d87-r78g7\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") " pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.426732 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.426692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:11.703419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.703385    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:39:11.704512 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:39:11.704485    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ecf5b1c_b910_4eb6_b9bc_098518d6093b.slice/crio-550bd3d19b11dfe0c39a2b2b57853971e73a2be36eb8b213798ed64906265161 WatchSource:0}: Error finding container 550bd3d19b11dfe0c39a2b2b57853971e73a2be36eb8b213798ed64906265161: Status 404 returned error can't find the container with id 550bd3d19b11dfe0c39a2b2b57853971e73a2be36eb8b213798ed64906265161
Sep 12 00:39:11.895852 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.895813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" event={"ID":"0ecf5b1c-b910-4eb6-b9bc-098518d6093b","Type":"ContainerStarted","Data":"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"}
Sep 12 00:39:11.896036 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.895868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" event={"ID":"0ecf5b1c-b910-4eb6-b9bc-098518d6093b","Type":"ContainerStarted","Data":"550bd3d19b11dfe0c39a2b2b57853971e73a2be36eb8b213798ed64906265161"}
Sep 12 00:39:11.912388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:11.912329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" podStartSLOduration=0.912308144 podStartE2EDuration="912.308144ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:11.91040333 +0000 UTC m=+1348.044393556" watchObservedRunningTime="2026-09-12 00:39:11.912308144 +0000 UTC m=+1348.046298371"
Sep 12 00:39:17.121916 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.121877    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:39:17.125378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.125359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.127825 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.127801    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-0d3a66\""
Sep 12 00:39:17.127925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.127848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-0d3a66-dockercfg-t9gbt\""
Sep 12 00:39:17.136756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.136732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:39:17.145818 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.145793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.145914 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.145873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgtxk\" (UniqueName: \"kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.246484 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.246447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgtxk\" (UniqueName: \"kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.246673 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.246561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.247285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.247266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.254166 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.254139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgtxk\" (UniqueName: \"kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk\") pod \"payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.437046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.437013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.711691 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.711662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:39:17.713342 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:39:17.713319    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfa2ba7_223c_4b8d_a823_459c6ffa37d1.slice/crio-5ee8d43d0ff90c7f46d36558190c1e97f442887c233c7ef70fbc73508e87a2ad WatchSource:0}: Error finding container 5ee8d43d0ff90c7f46d36558190c1e97f442887c233c7ef70fbc73508e87a2ad: Status 404 returned error can't find the container with id 5ee8d43d0ff90c7f46d36558190c1e97f442887c233c7ef70fbc73508e87a2ad
Sep 12 00:39:17.930436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.930400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" event={"ID":"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1","Type":"ContainerStarted","Data":"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"}
Sep 12 00:39:17.930436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.930451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" event={"ID":"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1","Type":"ContainerStarted","Data":"5ee8d43d0ff90c7f46d36558190c1e97f442887c233c7ef70fbc73508e87a2ad"}
Sep 12 00:39:17.930699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.930499    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:17.947963 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:17.947914    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" podStartSLOduration=0.947899367 podStartE2EDuration="947.899367ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:17.945160859 +0000 UTC m=+1354.079151085" watchObservedRunningTime="2026-09-12 00:39:17.947899367 +0000 UTC m=+1354.081889592"
Sep 12 00:39:21.274577 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.274546    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"]
Sep 12 00:39:21.278032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.278014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:21.280265 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.280244    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:39:21.285535 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.285511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"]
Sep 12 00:39:21.388076 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.388036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql\" (UID: \"47a4f390-6507-43fd-9dd0-6e187b66dac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:21.427580 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.427539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:21.427824 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.427597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:21.440232 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.440207    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:21.488814 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.488754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql\" (UID: \"47a4f390-6507-43fd-9dd0-6e187b66dac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:21.489508 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.489489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql\" (UID: \"47a4f390-6507-43fd-9dd0-6e187b66dac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:21.590990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.590955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:21.862612 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.862581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"]
Sep 12 00:39:21.863890 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:39:21.863855    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a4f390_6507_43fd_9dd0_6e187b66dac4.slice/crio-ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb WatchSource:0}: Error finding container ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb: Status 404 returned error can't find the container with id ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb
Sep 12 00:39:21.954988 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.954948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql" event={"ID":"47a4f390-6507-43fd-9dd0-6e187b66dac4","Type":"ContainerStarted","Data":"ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb"}
Sep 12 00:39:21.966199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:21.966173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:39:22.962534 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:22.962498    2580 generic.go:356] "Generic (PLEG): container finished" podID="47a4f390-6507-43fd-9dd0-6e187b66dac4" containerID="4d2a8bc55f7bf6b61e72337f1247259c2065ff13950262fc1663b4a30764344e" exitCode=0
Sep 12 00:39:22.963057 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:22.962596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql" event={"ID":"47a4f390-6507-43fd-9dd0-6e187b66dac4","Type":"ContainerDied","Data":"4d2a8bc55f7bf6b61e72337f1247259c2065ff13950262fc1663b4a30764344e"}
Sep 12 00:39:24.101711 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.101686    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:24.219815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.219694    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca\") pod \"47a4f390-6507-43fd-9dd0-6e187b66dac4\" (UID: \"47a4f390-6507-43fd-9dd0-6e187b66dac4\") "
Sep 12 00:39:24.220110 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.220085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47a4f390-6507-43fd-9dd0-6e187b66dac4" (UID: "47a4f390-6507-43fd-9dd0-6e187b66dac4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:39:24.320864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.320826    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a4f390-6507-43fd-9dd0-6e187b66dac4-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:24.975462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.975431    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"
Sep 12 00:39:24.975634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.975430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql" event={"ID":"47a4f390-6507-43fd-9dd0-6e187b66dac4","Type":"ContainerDied","Data":"ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb"}
Sep 12 00:39:24.975634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:24.975558    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab149e79787ad1aacd1cd3e124240cde26a38d9a56e3cb87048d29ac8d4ceacb"
Sep 12 00:39:28.939328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:28.939290    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:39:32.268363 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:32.268325    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"]
Sep 12 00:39:32.270794 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:32.270744    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a4df66-zwqql"]
Sep 12 00:39:32.381940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:32.381905    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47a4f390-6507-43fd-9dd0-6e187b66dac4" path="/var/lib/kubelet/pods/47a4f390-6507-43fd-9dd0-6e187b66dac4/volumes"
Sep 12 00:39:37.641662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:37.641621    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:39:47.844921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:47.844890    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:39:52.661219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:52.661155    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" podUID="586e7728-8437-49fd-a92b-6b425c115945" containerName="main" containerID="cri-o://0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0" gracePeriod=45
Sep 12 00:39:52.918792 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:52.918712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:39:53.026421 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pk4w\" (UniqueName: \"kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026446    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026536    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026567    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026592    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home\") pod \"586e7728-8437-49fd-a92b-6b425c115945\" (UID: \"586e7728-8437-49fd-a92b-6b425c115945\") "
Sep 12 00:39:53.026886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache" (OuterVolumeSpecName: "model-cache") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:53.026886 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:53.027014 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.026991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home" (OuterVolumeSpecName: "home") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:53.027069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.027007    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.027069 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.027028    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.028673 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.028647    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm" (OuterVolumeSpecName: "dshm") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:53.028817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.028789    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w" (OuterVolumeSpecName: "kube-api-access-5pk4w") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "kube-api-access-5pk4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:53.029077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.029053    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "586e7728-8437-49fd-a92b-6b425c115945" (UID: "586e7728-8437-49fd-a92b-6b425c115945"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:39:53.128357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.128318    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pk4w\" (UniqueName: \"kubernetes.io/projected/586e7728-8437-49fd-a92b-6b425c115945-kube-api-access-5pk4w\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.128357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.128354    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.128357 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.128367    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/586e7728-8437-49fd-a92b-6b425c115945-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.128595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.128377    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/586e7728-8437-49fd-a92b-6b425c115945-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:39:53.136374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.136334    2580 generic.go:356] "Generic (PLEG): container finished" podID="586e7728-8437-49fd-a92b-6b425c115945" containerID="0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0" exitCode=0
Sep 12 00:39:53.136520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.136375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" event={"ID":"586e7728-8437-49fd-a92b-6b425c115945","Type":"ContainerDied","Data":"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"}
Sep 12 00:39:53.136520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.136421    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"
Sep 12 00:39:53.136520 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.136436    2580 scope.go:117] "RemoveContainer" containerID="0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"
Sep 12 00:39:53.136688 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.136426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls" event={"ID":"586e7728-8437-49fd-a92b-6b425c115945","Type":"ContainerDied","Data":"5c0a989c90116aa3bd449ed947f2791a0853fe9fa00ec2689a28514cb4ed249f"}
Sep 12 00:39:53.147431 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.147414    2580 scope.go:117] "RemoveContainer" containerID="0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"
Sep 12 00:39:53.147712 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:39:53.147695    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0\": container with ID starting with 0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0 not found: ID does not exist" containerID="0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"
Sep 12 00:39:53.147779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.147721    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0"} err="failed to get container status \"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0\": rpc error: code = NotFound desc = could not find container \"0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0\": container with ID starting with 0c87f5b82d794128ef6596eea2210b00c38cd2763e2284dd4d51ea2c25ea54d0 not found: ID does not exist"
Sep 12 00:39:53.159624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.159598    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:39:53.163178 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:53.163151    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a4df66/test-model-a4df66-kserve-74bbbb47f5-wmnls"]
Sep 12 00:39:54.375643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:39:54.375610    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="586e7728-8437-49fd-a92b-6b425c115945" path="/var/lib/kubelet/pods/586e7728-8437-49fd-a92b-6b425c115945/volumes"
Sep 12 00:40:02.864617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:02.864574    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" podUID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" containerName="main" containerID="cri-o://cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313" gracePeriod=45
Sep 12 00:40:03.112546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.112516    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:40:03.196747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.196713    2580 generic.go:356] "Generic (PLEG): container finished" podID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" containerID="cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313" exitCode=0
Sep 12 00:40:03.196747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.196752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" event={"ID":"0ecf5b1c-b910-4eb6-b9bc-098518d6093b","Type":"ContainerDied","Data":"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"}
Sep 12 00:40:03.196982 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.196755    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"
Sep 12 00:40:03.196982 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.196795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7" event={"ID":"0ecf5b1c-b910-4eb6-b9bc-098518d6093b","Type":"ContainerDied","Data":"550bd3d19b11dfe0c39a2b2b57853971e73a2be36eb8b213798ed64906265161"}
Sep 12 00:40:03.196982 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.196812    2580 scope.go:117] "RemoveContainer" containerID="cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"
Sep 12 00:40:03.206018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.205994    2580 scope.go:117] "RemoveContainer" containerID="cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"
Sep 12 00:40:03.206268 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:03.206251    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313\": container with ID starting with cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313 not found: ID does not exist" containerID="cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"
Sep 12 00:40:03.206328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.206278    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313"} err="failed to get container status \"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313\": rpc error: code = NotFound desc = could not find container \"cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313\": container with ID starting with cd642a62cec6f80312c5d7e5a9a9715c2a00efdf42146d17f947458c53b88313 not found: ID does not exist"
Sep 12 00:40:03.231140 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jl4k\" (UniqueName: \"kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231303    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231346    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs\") pod \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\" (UID: \"0ecf5b1c-b910-4eb6-b9bc-098518d6093b\") "
Sep 12 00:40:03.231491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231369    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home" (OuterVolumeSpecName: "home") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:03.231491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:03.231704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:03.231743 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231719    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.231743 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231733    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.231743 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.231742    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.233646 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.233620    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k" (OuterVolumeSpecName: "kube-api-access-9jl4k") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "kube-api-access-9jl4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:03.233756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.233652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm" (OuterVolumeSpecName: "dshm") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:03.233756 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.233651    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ecf5b1c-b910-4eb6-b9bc-098518d6093b" (UID: "0ecf5b1c-b910-4eb6-b9bc-098518d6093b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:40:03.333217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.333174    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.333217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.333208    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jl4k\" (UniqueName: \"kubernetes.io/projected/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-kube-api-access-9jl4k\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.333217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.333218    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ecf5b1c-b910-4eb6-b9bc-098518d6093b-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:03.520452 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.520419    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:40:03.522821 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:03.522787    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-0a0781/noten-model-0a0781-kserve-6c6bdd9d87-r78g7"]
Sep 12 00:40:04.375996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:04.375964    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" path="/var/lib/kubelet/pods/0ecf5b1c-b910-4eb6-b9bc-098518d6093b/volumes"
Sep 12 00:40:12.920226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920192    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:40:12.920611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920588    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" containerName="main"
Sep 12 00:40:12.920611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920600    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" containerName="main"
Sep 12 00:40:12.920684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920617    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47a4f390-6507-43fd-9dd0-6e187b66dac4" containerName="revoke-keys"
Sep 12 00:40:12.920684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920623    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="47a4f390-6507-43fd-9dd0-6e187b66dac4" containerName="revoke-keys"
Sep 12 00:40:12.920684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920630    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586e7728-8437-49fd-a92b-6b425c115945" containerName="main"
Sep 12 00:40:12.920684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920635    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="586e7728-8437-49fd-a92b-6b425c115945" containerName="main"
Sep 12 00:40:12.920862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920701    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="47a4f390-6507-43fd-9dd0-6e187b66dac4" containerName="revoke-keys"
Sep 12 00:40:12.920862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920711    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ecf5b1c-b910-4eb6-b9bc-098518d6093b" containerName="main"
Sep 12 00:40:12.920862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.920718    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="586e7728-8437-49fd-a92b-6b425c115945" containerName="main"
Sep 12 00:40:12.925339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.925313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:12.927787 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.927745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"e2e-conflict-b-40370e1e-kserve-self-signed-certs\""
Sep 12 00:40:12.933730 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:12.933709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:40:13.030939 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.030853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.030939 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.030925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.031192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.030967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.031192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.030984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.031192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.031002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.031192 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.031026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64dcf\" (UniqueName: \"kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.131952 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.131916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.131974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64dcf\" (UniqueName: \"kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132484 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132571 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.132571 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.132534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.134350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.134329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.134731 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.134710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.139985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.139962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64dcf\" (UniqueName: \"kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf\") pod \"e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") " pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.236256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.236163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:13.509127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.509100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:40:13.510622 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:13.510588    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfca3fd9c_c36f_4885_a456_3186d904e25d.slice/crio-b589d5ab3335e60f711ef50df64cfe81b3a7626995b112d8610bffae0d9881e0 WatchSource:0}: Error finding container b589d5ab3335e60f711ef50df64cfe81b3a7626995b112d8610bffae0d9881e0: Status 404 returned error can't find the container with id b589d5ab3335e60f711ef50df64cfe81b3a7626995b112d8610bffae0d9881e0
Sep 12 00:40:13.540338 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.540306    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7882-e7fef7"]
Sep 12 00:40:13.544911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.544894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:13.547314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.547291    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qwc4m\""
Sep 12 00:40:13.549713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.549687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-e7fef7"]
Sep 12 00:40:13.737155 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.737120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9\") pod \"test-curl-7882-e7fef7\" (UID: \"8a69c070-cda8-4407-91d5-c9aed6d07f02\") " pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:13.837987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.837912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9\") pod \"test-curl-7882-e7fef7\" (UID: \"8a69c070-cda8-4407-91d5-c9aed6d07f02\") " pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:13.845252 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.845220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9\") pod \"test-curl-7882-e7fef7\" (UID: \"8a69c070-cda8-4407-91d5-c9aed6d07f02\") " pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:13.868221 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:13.868188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:14.136186 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.136159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-e7fef7"]
Sep 12 00:40:14.138487 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:14.138461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a69c070_cda8_4407_91d5_c9aed6d07f02.slice/crio-6298fb0a16da3ec18795b41bb01d6eeeb8165987919bdae046af6092c6965f06 WatchSource:0}: Error finding container 6298fb0a16da3ec18795b41bb01d6eeeb8165987919bdae046af6092c6965f06: Status 404 returned error can't find the container with id 6298fb0a16da3ec18795b41bb01d6eeeb8165987919bdae046af6092c6965f06
Sep 12 00:40:14.265267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.265234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" event={"ID":"fca3fd9c-c36f-4885-a456-3186d904e25d","Type":"ContainerStarted","Data":"39ef3d20b87d73a33cd564cdd31d7b6ab38a7ab0d6aa2b846ef1a00fd47dbd89"}
Sep 12 00:40:14.265456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.265278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" event={"ID":"fca3fd9c-c36f-4885-a456-3186d904e25d","Type":"ContainerStarted","Data":"b589d5ab3335e60f711ef50df64cfe81b3a7626995b112d8610bffae0d9881e0"}
Sep 12 00:40:14.266703 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.266678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-e7fef7" event={"ID":"8a69c070-cda8-4407-91d5-c9aed6d07f02","Type":"ContainerStarted","Data":"d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a"}
Sep 12 00:40:14.266830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.266711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-e7fef7" event={"ID":"8a69c070-cda8-4407-91d5-c9aed6d07f02","Type":"ContainerStarted","Data":"6298fb0a16da3ec18795b41bb01d6eeeb8165987919bdae046af6092c6965f06"}
Sep 12 00:40:14.283053 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.283001    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" podStartSLOduration=2.2829852170000002 podStartE2EDuration="2.282985217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:14.280642105 +0000 UTC m=+1410.414632331" watchObservedRunningTime="2026-09-12 00:40:14.282985217 +0000 UTC m=+1410.416975494"
Sep 12 00:40:14.296436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.296383    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7882-e7fef7" podStartSLOduration=1.2963632 podStartE2EDuration="1.2963632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:14.293189059 +0000 UTC m=+1410.427179284" watchObservedRunningTime="2026-09-12 00:40:14.2963632 +0000 UTC m=+1410.430353425"
Sep 12 00:40:14.886972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.886864    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7882-e7fef7"]
Sep 12 00:40:14.895483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:14.892859    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7882-e7fef7"]
Sep 12 00:40:15.020802 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.020750    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7882-ecc5ad"]
Sep 12 00:40:15.021236 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.021216    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" containerName="test-curl-7882-e7fef7"
Sep 12 00:40:15.021349 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.021239    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" containerName="test-curl-7882-e7fef7"
Sep 12 00:40:15.021406 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.021364    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" containerName="test-curl-7882-e7fef7"
Sep 12 00:40:15.024518 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.024497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:15.030231 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.030203    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-ecc5ad"]
Sep 12 00:40:15.150568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.150474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7tfm\" (UniqueName: \"kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm\") pod \"test-curl-7882-ecc5ad\" (UID: \"adc18dd1-bd89-4955-bda9-7bd42bd1924a\") " pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:15.251222 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.251178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7tfm\" (UniqueName: \"kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm\") pod \"test-curl-7882-ecc5ad\" (UID: \"adc18dd1-bd89-4955-bda9-7bd42bd1924a\") " pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:15.258776 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.258740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7tfm\" (UniqueName: \"kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm\") pod \"test-curl-7882-ecc5ad\" (UID: \"adc18dd1-bd89-4955-bda9-7bd42bd1924a\") " pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:15.273289 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:15.273260    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7882-e7fef7" container="test-curl-7882-e7fef7"
Sep 12 00:40:15.335790 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.335737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:15.607365 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:15.607334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-ecc5ad"]
Sep 12 00:40:15.610581 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:15.610556    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadc18dd1_bd89_4955_bda9_7bd42bd1924a.slice/crio-ae0a3fe7733b5928b9f2cef9fa647c4f2612d39eeedb081c0e3622397623dc4b WatchSource:0}: Error finding container ae0a3fe7733b5928b9f2cef9fa647c4f2612d39eeedb081c0e3622397623dc4b: Status 404 returned error can't find the container with id ae0a3fe7733b5928b9f2cef9fa647c4f2612d39eeedb081c0e3622397623dc4b
Sep 12 00:40:16.280218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.280183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-ecc5ad" event={"ID":"adc18dd1-bd89-4955-bda9-7bd42bd1924a","Type":"ContainerStarted","Data":"8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c"}
Sep 12 00:40:16.280218 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.280226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-ecc5ad" event={"ID":"adc18dd1-bd89-4955-bda9-7bd42bd1924a","Type":"ContainerStarted","Data":"ae0a3fe7733b5928b9f2cef9fa647c4f2612d39eeedb081c0e3622397623dc4b"}
Sep 12 00:40:16.280721 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.280355    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7882-e7fef7" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" containerName="test-curl-7882-e7fef7" containerID="cri-o://d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a" gracePeriod=2
Sep 12 00:40:16.294265 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.294223    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7882-ecc5ad" podStartSLOduration=1.294208082 podStartE2EDuration="1.294208082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:16.292460969 +0000 UTC m=+1412.426451193" watchObservedRunningTime="2026-09-12 00:40:16.294208082 +0000 UTC m=+1412.428198304"
Sep 12 00:40:16.294410 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:16.294233    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-e7fef7\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:16.903317 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.903277    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7882-ecc5ad"]
Sep 12 00:40:16.911460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:16.911431    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7882-ecc5ad"]
Sep 12 00:40:17.056930 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.056852    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7882-142526"]
Sep 12 00:40:17.057252 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.057238    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" containerName="test-curl-7882-ecc5ad"
Sep 12 00:40:17.057301 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.057254    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" containerName="test-curl-7882-ecc5ad"
Sep 12 00:40:17.057348 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.057339    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" containerName="test-curl-7882-ecc5ad"
Sep 12 00:40:17.060272 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.060255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:17.066662 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.066529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv6vw\" (UniqueName: \"kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw\") pod \"test-curl-7882-142526\" (UID: \"789fcaa1-ff0c-4712-84a5-ea612ada6ff1\") " pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:17.072136 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.072114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-142526"]
Sep 12 00:40:17.166990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.166959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv6vw\" (UniqueName: \"kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw\") pod \"test-curl-7882-142526\" (UID: \"789fcaa1-ff0c-4712-84a5-ea612ada6ff1\") " pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:17.173784 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.173751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv6vw\" (UniqueName: \"kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw\") pod \"test-curl-7882-142526\" (UID: \"789fcaa1-ff0c-4712-84a5-ea612ada6ff1\") " pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:17.285835 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:17.285809    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7882-ecc5ad" container="test-curl-7882-ecc5ad"
Sep 12 00:40:17.369384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.369302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:17.638583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:17.638553    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-142526"]
Sep 12 00:40:17.640024 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:17.639996    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod789fcaa1_ff0c_4712_84a5_ea612ada6ff1.slice/crio-9038ce13daf314a3ad2fea9e6ea5c2e5d47ffe58006bc813e42f85f0940ccfb6 WatchSource:0}: Error finding container 9038ce13daf314a3ad2fea9e6ea5c2e5d47ffe58006bc813e42f85f0940ccfb6: Status 404 returned error can't find the container with id 9038ce13daf314a3ad2fea9e6ea5c2e5d47ffe58006bc813e42f85f0940ccfb6
Sep 12 00:40:18.293866 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.293834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-142526" event={"ID":"789fcaa1-ff0c-4712-84a5-ea612ada6ff1","Type":"ContainerStarted","Data":"22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484"}
Sep 12 00:40:18.294204 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.293881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-142526" event={"ID":"789fcaa1-ff0c-4712-84a5-ea612ada6ff1","Type":"ContainerStarted","Data":"9038ce13daf314a3ad2fea9e6ea5c2e5d47ffe58006bc813e42f85f0940ccfb6"}
Sep 12 00:40:18.294204 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.294028    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7882-ecc5ad" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" containerName="test-curl-7882-ecc5ad" containerID="cri-o://8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c" gracePeriod=2
Sep 12 00:40:18.307563 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:18.307530    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-ecc5ad\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:18.308478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.308437    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7882-142526" podStartSLOduration=1.308423302 podStartE2EDuration="1.308423302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:18.305916736 +0000 UTC m=+1414.439906962" watchObservedRunningTime="2026-09-12 00:40:18.308423302 +0000 UTC m=+1414.442413527"
Sep 12 00:40:18.463058 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.463035    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:18.480754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.480734    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9\") pod \"8a69c070-cda8-4407-91d5-c9aed6d07f02\" (UID: \"8a69c070-cda8-4407-91d5-c9aed6d07f02\") "
Sep 12 00:40:18.482779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.482742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9" (OuterVolumeSpecName: "kube-api-access-p92x9") pod "8a69c070-cda8-4407-91d5-c9aed6d07f02" (UID: "8a69c070-cda8-4407-91d5-c9aed6d07f02"). InnerVolumeSpecName "kube-api-access-p92x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:18.582168 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.582082    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p92x9\" (UniqueName: \"kubernetes.io/projected/8a69c070-cda8-4407-91d5-c9aed6d07f02-kube-api-access-p92x9\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:18.883486 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.883401    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7882-142526"]
Sep 12 00:40:18.887388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.887363    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7882-142526"]
Sep 12 00:40:18.999913 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:18.999883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7882-860d1f"]
Sep 12 00:40:19.000253 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.000242    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" containerName="test-curl-7882-142526"
Sep 12 00:40:19.000293 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.000255    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" containerName="test-curl-7882-142526"
Sep 12 00:40:19.000342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.000334    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" containerName="test-curl-7882-142526"
Sep 12 00:40:19.003376 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.003357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:19.007409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.007382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-860d1f"]
Sep 12 00:40:19.087950 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.087914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx94f\" (UniqueName: \"kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f\") pod \"test-curl-7882-860d1f\" (UID: \"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd\") " pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:19.189350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.189315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx94f\" (UniqueName: \"kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f\") pod \"test-curl-7882-860d1f\" (UID: \"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd\") " pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:19.197243 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.197216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx94f\" (UniqueName: \"kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f\") pod \"test-curl-7882-860d1f\" (UID: \"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd\") " pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:19.300988 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.300955    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a69c070-cda8-4407-91d5-c9aed6d07f02" containerID="d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a" exitCode=137
Sep 12 00:40:19.301380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.300995    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:19.301380 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.301046    2580 scope.go:117] "RemoveContainer" containerID="d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a"
Sep 12 00:40:19.301380 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:19.301262    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7882-142526" container="test-curl-7882-142526"
Sep 12 00:40:19.310872 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.310854    2580 scope.go:117] "RemoveContainer" containerID="d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a"
Sep 12 00:40:19.311187 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:19.311169    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a\": container with ID starting with d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a not found: ID does not exist" containerID="d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a"
Sep 12 00:40:19.311321 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.311198    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a"} err="failed to get container status \"d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a\": rpc error: code = NotFound desc = could not find container \"d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a\": container with ID starting with d6ae7aa0f19e39ca96e384eae67f6949665a702f016a3c1be377e54c736e8d0a not found: ID does not exist"
Sep 12 00:40:19.312137 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:19.312114    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-e7fef7\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:19.314286 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.314273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:19.589912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:19.589890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7882-860d1f"]
Sep 12 00:40:19.592014 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:19.591983    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24d7ffc7_4f9e_4f6f_bf3f_f2e728a2e3cd.slice/crio-f89f13167056d28726ca83e47535d520bd97c5b8dc4e22bf073a45a7dc518f5b WatchSource:0}: Error finding container f89f13167056d28726ca83e47535d520bd97c5b8dc4e22bf073a45a7dc518f5b: Status 404 returned error can't find the container with id f89f13167056d28726ca83e47535d520bd97c5b8dc4e22bf073a45a7dc518f5b
Sep 12 00:40:20.309256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.309201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-860d1f" event={"ID":"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd","Type":"ContainerStarted","Data":"185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a"}
Sep 12 00:40:20.309606 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.309259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7882-860d1f" event={"ID":"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd","Type":"ContainerStarted","Data":"f89f13167056d28726ca83e47535d520bd97c5b8dc4e22bf073a45a7dc518f5b"}
Sep 12 00:40:20.309606 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.309261    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7882-142526" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" containerName="test-curl-7882-142526" containerID="cri-o://22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484" gracePeriod=2
Sep 12 00:40:20.311282 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:20.311253    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:20.312881 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:20.312858    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-e7fef7\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:20.327088 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:20.327060    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:20.327927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.327876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7882-860d1f" podStartSLOduration=2.32785762 podStartE2EDuration="2.32785762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:20.324677454 +0000 UTC m=+1416.458667679" watchObservedRunningTime="2026-09-12 00:40:20.32785762 +0000 UTC m=+1416.461847846"
Sep 12 00:40:20.328581 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:20.328559    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-e7fef7\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" pod="opendatahub/test-curl-7882-e7fef7"
Sep 12 00:40:20.376307 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.376270    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a69c070-cda8-4407-91d5-c9aed6d07f02" path="/var/lib/kubelet/pods/8a69c070-cda8-4407-91d5-c9aed6d07f02/volumes"
Sep 12 00:40:20.474579 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.474551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:20.503623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.503596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7tfm\" (UniqueName: \"kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm\") pod \"adc18dd1-bd89-4955-bda9-7bd42bd1924a\" (UID: \"adc18dd1-bd89-4955-bda9-7bd42bd1924a\") "
Sep 12 00:40:20.505968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.505938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm" (OuterVolumeSpecName: "kube-api-access-s7tfm") pod "adc18dd1-bd89-4955-bda9-7bd42bd1924a" (UID: "adc18dd1-bd89-4955-bda9-7bd42bd1924a"). InnerVolumeSpecName "kube-api-access-s7tfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:20.605353 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:20.605275    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7tfm\" (UniqueName: \"kubernetes.io/projected/adc18dd1-bd89-4955-bda9-7bd42bd1924a-kube-api-access-s7tfm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:21.002461 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.002420    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7882-860d1f"]
Sep 12 00:40:21.008750 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.008723    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7882-860d1f"]
Sep 12 00:40:21.017263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.017222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"]
Sep 12 00:40:21.017878 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.017849    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" containerName="test-curl-7882-860d1f"
Sep 12 00:40:21.017878 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.017876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" containerName="test-curl-7882-860d1f"
Sep 12 00:40:21.018094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.018000    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" containerName="test-curl-7882-860d1f"
Sep 12 00:40:21.021096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.021080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:21.023663 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.023644    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:40:21.026811 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.026786    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"]
Sep 12 00:40:21.110444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.110400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9\" (UID: \"0a57c489-0f1d-4882-9d15-2fb0957ddeb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:21.212031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.211991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9\" (UID: \"0a57c489-0f1d-4882-9d15-2fb0957ddeb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:21.212622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.212601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9\" (UID: \"0a57c489-0f1d-4882-9d15-2fb0957ddeb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:21.316410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.316319    2580 generic.go:356] "Generic (PLEG): container finished" podID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" containerID="8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c" exitCode=137
Sep 12 00:40:21.316410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.316384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:21.316959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.316426    2580 scope.go:117] "RemoveContainer" containerID="8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c"
Sep 12 00:40:21.316959 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:21.316792    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7882-860d1f" container="test-curl-7882-860d1f"
Sep 12 00:40:21.326202 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.326186    2580 scope.go:117] "RemoveContainer" containerID="8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c"
Sep 12 00:40:21.326457 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:21.326438    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c\": container with ID starting with 8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c not found: ID does not exist" containerID="8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c"
Sep 12 00:40:21.326503 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.326465    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c"} err="failed to get container status \"8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c\": rpc error: code = NotFound desc = could not find container \"8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c\": container with ID starting with 8e80f6f3e6d44742d071b28158c93be8c72931c25458433e7c5403f5b80f124c not found: ID does not exist"
Sep 12 00:40:21.326954 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:21.326932    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-ecc5ad\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:21.335273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.335256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:21.613455 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.613421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"]
Sep 12 00:40:21.613652 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:21.613625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a57c489_0f1d_4882_9d15_2fb0957ddeb1.slice/crio-cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86 WatchSource:0}: Error finding container cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86: Status 404 returned error can't find the container with id cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86
Sep 12 00:40:21.907893 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.907859    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:40:21.912687 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.912662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:21.914667 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:21.914622    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-ecc5ad\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:21.914997 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.914975    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cfdf5d6c-openshift-default-dockercfg-4ptrq\""
Sep 12 00:40:21.923526 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:21.923503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:40:22.019831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfmd\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.019993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.020092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.020136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.020304 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.020180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121478 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfmd\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121758 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.121974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.121961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.122276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.122259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.123589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.123572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.123878 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.123860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.129968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.129936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.130119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.130103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfmd\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd\") pod \"e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") " pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.227431 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.227398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:22.325123 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.325091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9" event={"ID":"0a57c489-0f1d-4882-9d15-2fb0957ddeb1","Type":"ContainerStarted","Data":"776a3495fefa19f1e08618b8f2a8e2061c9f28225cd8c81124f7148e874eb4d8"}
Sep 12 00:40:22.325510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.325143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9" event={"ID":"0a57c489-0f1d-4882-9d15-2fb0957ddeb1","Type":"ContainerStarted","Data":"cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86"}
Sep 12 00:40:22.325510 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.325135    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7882-860d1f" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" containerName="test-curl-7882-860d1f" containerID="cri-o://185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a" gracePeriod=2
Sep 12 00:40:22.327085 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:22.327059    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:22.340988 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.340937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9" podStartSLOduration=2.340919777 podStartE2EDuration="2.340919777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:22.340266357 +0000 UTC m=+1418.474256583" watchObservedRunningTime="2026-09-12 00:40:22.340919777 +0000 UTC m=+1418.474910002"
Sep 12 00:40:22.341903 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:22.341878    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-ecc5ad\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:22.343834 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:22.343809    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:22.345258 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:22.345240    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-ecc5ad\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" pod="opendatahub/test-curl-7882-ecc5ad"
Sep 12 00:40:22.379246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.379165    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc18dd1-bd89-4955-bda9-7bd42bd1924a" path="/var/lib/kubelet/pods/adc18dd1-bd89-4955-bda9-7bd42bd1924a/volumes"
Sep 12 00:40:22.477446 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.477412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:22.503637 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.503610    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:40:22.504915 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:22.504887    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aecb8de_0f4c_4d54_8668_0839e7dfbcbd.slice/crio-77a2a2c18ab9a85581a93f3c8e031dac2c426d8312ead9c583d648b5f51c4175 WatchSource:0}: Error finding container 77a2a2c18ab9a85581a93f3c8e031dac2c426d8312ead9c583d648b5f51c4175: Status 404 returned error can't find the container with id 77a2a2c18ab9a85581a93f3c8e031dac2c426d8312ead9c583d648b5f51c4175
Sep 12 00:40:22.507207 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.507175    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:22.507283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.507242    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:22.507283 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.507267    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:22.525334 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.525313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv6vw\" (UniqueName: \"kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw\") pod \"789fcaa1-ff0c-4712-84a5-ea612ada6ff1\" (UID: \"789fcaa1-ff0c-4712-84a5-ea612ada6ff1\") "
Sep 12 00:40:22.527176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.527155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw" (OuterVolumeSpecName: "kube-api-access-wv6vw") pod "789fcaa1-ff0c-4712-84a5-ea612ada6ff1" (UID: "789fcaa1-ff0c-4712-84a5-ea612ada6ff1"). InnerVolumeSpecName "kube-api-access-wv6vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:22.626583 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:22.626548    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv6vw\" (UniqueName: \"kubernetes.io/projected/789fcaa1-ff0c-4712-84a5-ea612ada6ff1-kube-api-access-wv6vw\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:23.236596 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.236561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:23.236801 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.236615    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:23.249607 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.249574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:23.332388 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.332347    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a57c489-0f1d-4882-9d15-2fb0957ddeb1" containerID="776a3495fefa19f1e08618b8f2a8e2061c9f28225cd8c81124f7148e874eb4d8" exitCode=0
Sep 12 00:40:23.332863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.332447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9" event={"ID":"0a57c489-0f1d-4882-9d15-2fb0957ddeb1","Type":"ContainerDied","Data":"776a3495fefa19f1e08618b8f2a8e2061c9f28225cd8c81124f7148e874eb4d8"}
Sep 12 00:40:23.333726 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.333705    2580 generic.go:356] "Generic (PLEG): container finished" podID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" containerID="22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484" exitCode=137
Sep 12 00:40:23.333867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.333737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:23.333867 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.333807    2580 scope.go:117] "RemoveContainer" containerID="22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484"
Sep 12 00:40:23.335560 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.335539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerStarted","Data":"8ba01883fdd11f2be621669db7b114b0d241203fc274bd1267f5c87ca76ba3fa"}
Sep 12 00:40:23.335657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.335584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerStarted","Data":"77a2a2c18ab9a85581a93f3c8e031dac2c426d8312ead9c583d648b5f51c4175"}
Sep 12 00:40:23.344908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.344891    2580 scope.go:117] "RemoveContainer" containerID="22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484"
Sep 12 00:40:23.345189 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:23.345168    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484\": container with ID starting with 22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484 not found: ID does not exist" containerID="22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484"
Sep 12 00:40:23.345233 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.345199    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484"} err="failed to get container status \"22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484\": rpc error: code = NotFound desc = could not find container \"22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484\": container with ID starting with 22187acac5c14f78d21b13e5762792c860f0a2cac30c1b7a5fa38a355d20c484 not found: ID does not exist"
Sep 12 00:40:23.349731 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.349704    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:40:23.364410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:23.364358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podStartSLOduration=2.364340796 podStartE2EDuration="2.364340796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:23.36349085 +0000 UTC m=+1419.497481077" watchObservedRunningTime="2026-09-12 00:40:23.364340796 +0000 UTC m=+1419.498331068"
Sep 12 00:40:23.365138 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:23.365109    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:23.383598 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:23.383559    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:24.227949 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.227911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:24.233507 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.233470    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:24.235256 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:24.235227    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:24.343293 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.342994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:24.343920 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.343901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:24.364841 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:24.364658    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:24.376236 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:24.376038    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-142526\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" pod="opendatahub/test-curl-7882-142526"
Sep 12 00:40:24.381285 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.381249    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="789fcaa1-ff0c-4712-84a5-ea612ada6ff1" path="/var/lib/kubelet/pods/789fcaa1-ff0c-4712-84a5-ea612ada6ff1/volumes"
Sep 12 00:40:24.501861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.501835    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:24.534025 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.534001    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:24.535789 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:24.535751    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:24.554214 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.554184    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca\") pod \"0a57c489-0f1d-4882-9d15-2fb0957ddeb1\" (UID: \"0a57c489-0f1d-4882-9d15-2fb0957ddeb1\") "
Sep 12 00:40:24.554495 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.554471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a57c489-0f1d-4882-9d15-2fb0957ddeb1" (UID: "0a57c489-0f1d-4882-9d15-2fb0957ddeb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:40:24.655118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.655086    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx94f\" (UniqueName: \"kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f\") pod \"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd\" (UID: \"24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd\") "
Sep 12 00:40:24.655409 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.655393    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a57c489-0f1d-4882-9d15-2fb0957ddeb1-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:24.657153 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.657128    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f" (OuterVolumeSpecName: "kube-api-access-gx94f") pod "24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" (UID: "24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd"). InnerVolumeSpecName "kube-api-access-gx94f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:24.756561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:24.756487    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx94f\" (UniqueName: \"kubernetes.io/projected/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd-kube-api-access-gx94f\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:25.350290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.350263    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"
Sep 12 00:40:25.350723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.350295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9" event={"ID":"0a57c489-0f1d-4882-9d15-2fb0957ddeb1","Type":"ContainerDied","Data":"cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86"}
Sep 12 00:40:25.350723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.350343    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf23fa26a7e3d11c2d0e8dc743f4e8eaa62784d79fb06e5287b1fc0f201c4d86"
Sep 12 00:40:25.351665 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.351635    2580 generic.go:356] "Generic (PLEG): container finished" podID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" containerID="185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a" exitCode=137
Sep 12 00:40:25.351777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.351681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:25.351777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.351723    2580 scope.go:117] "RemoveContainer" containerID="185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a"
Sep 12 00:40:25.353909 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:25.353885    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:25.361752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.361730    2580 scope.go:117] "RemoveContainer" containerID="185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a"
Sep 12 00:40:25.362149 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:25.362120    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a\": container with ID starting with 185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a not found: ID does not exist" containerID="185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a"
Sep 12 00:40:25.362267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:25.362160    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a"} err="failed to get container status \"185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a\": rpc error: code = NotFound desc = could not find container \"185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a\": container with ID starting with 185ef3b20ea391ca43fd5628cff41c323a2bd1327843a251611e0deb402c9f7a not found: ID does not exist"
Sep 12 00:40:25.366084 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:25.366053    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:25.371416 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:25.371395    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7882-860d1f\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" pod="opendatahub/test-curl-7882-860d1f"
Sep 12 00:40:26.373436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:26.373391    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd" path="/var/lib/kubelet/pods/24d7ffc7-4f9e-4f6f-bf3f-f2e728a2e3cd/volumes"
Sep 12 00:40:31.421031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:31.420994    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"]
Sep 12 00:40:31.423404 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:31.423382    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39d42b-l6gr9"]
Sep 12 00:40:32.375099 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:32.375060    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a57c489-0f1d-4882-9d15-2fb0957ddeb1" path="/var/lib/kubelet/pods/0a57c489-0f1d-4882-9d15-2fb0957ddeb1/volumes"
Sep 12 00:40:36.774176 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:36.774087    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:40:48.013677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.013634    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"]
Sep 12 00:40:48.014314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.014291    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a57c489-0f1d-4882-9d15-2fb0957ddeb1" containerName="revoke-keys"
Sep 12 00:40:48.014369 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.014320    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a57c489-0f1d-4882-9d15-2fb0957ddeb1" containerName="revoke-keys"
Sep 12 00:40:48.014441 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.014429    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a57c489-0f1d-4882-9d15-2fb0957ddeb1" containerName="revoke-keys"
Sep 12 00:40:48.019622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.019597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:48.023331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.022281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:40:48.023331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.022378    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"]
Sep 12 00:40:48.100551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.100501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47\" (UID: \"e9c2059d-c9c1-425c-a037-85e8e05da7cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:48.202150 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.202105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47\" (UID: \"e9c2059d-c9c1-425c-a037-85e8e05da7cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:48.202782 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.202746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47\" (UID: \"e9c2059d-c9c1-425c-a037-85e8e05da7cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:48.333515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.333411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:48.609345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:48.609321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"]
Sep 12 00:40:48.642632 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:40:48.642604    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c2059d_c9c1_425c_a037_85e8e05da7cf.slice/crio-2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7 WatchSource:0}: Error finding container 2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7: Status 404 returned error can't find the container with id 2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7
Sep 12 00:40:49.499460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.499426    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9c2059d-c9c1-425c-a037-85e8e05da7cf" containerID="a3f23834cd3e6f3f25bc55e0c7f0340c0ac65e4c3f131825d232c89873f5e201" exitCode=0
Sep 12 00:40:49.499894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.499515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47" event={"ID":"e9c2059d-c9c1-425c-a037-85e8e05da7cf","Type":"ContainerDied","Data":"a3f23834cd3e6f3f25bc55e0c7f0340c0ac65e4c3f131825d232c89873f5e201"}
Sep 12 00:40:49.499894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.499566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47" event={"ID":"e9c2059d-c9c1-425c-a037-85e8e05da7cf","Type":"ContainerStarted","Data":"2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7"}
Sep 12 00:40:49.501214 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.501192    2580 generic.go:356] "Generic (PLEG): container finished" podID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerID="8ba01883fdd11f2be621669db7b114b0d241203fc274bd1267f5c87ca76ba3fa" exitCode=0
Sep 12 00:40:49.501329 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.501222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerDied","Data":"8ba01883fdd11f2be621669db7b114b0d241203fc274bd1267f5c87ca76ba3fa"}
Sep 12 00:40:49.501613 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.501600    2580 scope.go:117] "RemoveContainer" containerID="8ba01883fdd11f2be621669db7b114b0d241203fc274bd1267f5c87ca76ba3fa"
Sep 12 00:40:49.502830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.502804    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:49.502930 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.502877    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:49.502996 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:49.502931    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:40:50.509195 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:50.509158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerStarted","Data":"8788808e1885454f6a3b50083c6e5e0daca79d301d73ca81c691c097fb2ac136"}
Sep 12 00:40:50.685432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:50.685406    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:50.741238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:50.741196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca\") pod \"e9c2059d-c9c1-425c-a037-85e8e05da7cf\" (UID: \"e9c2059d-c9c1-425c-a037-85e8e05da7cf\") "
Sep 12 00:40:50.741589 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:50.741559    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9c2059d-c9c1-425c-a037-85e8e05da7cf" (UID: "e9c2059d-c9c1-425c-a037-85e8e05da7cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:40:50.741717 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:50.741656    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c2059d-c9c1-425c-a037-85e8e05da7cf-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:51.228295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.228256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:51.229197 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.229165    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 12 00:40:51.229432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.229397    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 12 00:40:51.517841 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.517577    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"
Sep 12 00:40:51.532685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.531846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47" event={"ID":"e9c2059d-c9c1-425c-a037-85e8e05da7cf","Type":"ContainerDied","Data":"2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7"}
Sep 12 00:40:51.532685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.531937    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a1d6b6022a26163dfb997aae6328f9094ace68ee0473d3e41efbc43dcecdaa7"
Sep 12 00:40:51.794559 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:51.794461    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" podUID="55b35d18-1403-49bb-ba94-e101d5e60333" containerName="main" containerID="cri-o://12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23" gracePeriod=45
Sep 12 00:40:52.087012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.086983    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:40:52.157359 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157321    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157430    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztg4d\" (UniqueName: \"kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157495    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157639    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home\") pod \"55b35d18-1403-49bb-ba94-e101d5e60333\" (UID: \"55b35d18-1403-49bb-ba94-e101d5e60333\") "
Sep 12 00:40:52.157817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:52.157817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157717    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache" (OuterVolumeSpecName: "model-cache") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:52.157987 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.157946    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home" (OuterVolumeSpecName: "home") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:52.158071 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.158050    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.158129 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.158077    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.158129 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.158088    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.159775 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.159742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm" (OuterVolumeSpecName: "dshm") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:52.160022 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.159993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:40:52.160116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.160044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d" (OuterVolumeSpecName: "kube-api-access-ztg4d") pod "55b35d18-1403-49bb-ba94-e101d5e60333" (UID: "55b35d18-1403-49bb-ba94-e101d5e60333"). InnerVolumeSpecName "kube-api-access-ztg4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:52.233717 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.233213    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 12 00:40:52.233717 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.233281    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 12 00:40:52.259595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.259557    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztg4d\" (UniqueName: \"kubernetes.io/projected/55b35d18-1403-49bb-ba94-e101d5e60333-kube-api-access-ztg4d\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.259595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.259590    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55b35d18-1403-49bb-ba94-e101d5e60333-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.259595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.259599    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55b35d18-1403-49bb-ba94-e101d5e60333-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:40:52.536026 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.535981    2580 generic.go:356] "Generic (PLEG): container finished" podID="55b35d18-1403-49bb-ba94-e101d5e60333" containerID="12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23" exitCode=0
Sep 12 00:40:52.536530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.536147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" event={"ID":"55b35d18-1403-49bb-ba94-e101d5e60333","Type":"ContainerDied","Data":"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"}
Sep 12 00:40:52.536530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.536188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7" event={"ID":"55b35d18-1403-49bb-ba94-e101d5e60333","Type":"ContainerDied","Data":"a19b08c6b82ef77bcac739007a666ecb8577157cf46e8a6a773e98fb96146b69"}
Sep 12 00:40:52.536530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.536212    2580 scope.go:117] "RemoveContainer" containerID="12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"
Sep 12 00:40:52.536530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.536384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"
Sep 12 00:40:52.549163 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.549141    2580 scope.go:117] "RemoveContainer" containerID="12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"
Sep 12 00:40:52.549415 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:40:52.549394    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23\": container with ID starting with 12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23 not found: ID does not exist" containerID="12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"
Sep 12 00:40:52.549477 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.549424    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23"} err="failed to get container status \"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23\": rpc error: code = NotFound desc = could not find container \"12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23\": container with ID starting with 12c9451c23f714c05277fa40463f83b1617a153fa72bfb51c8d73acfb7143c23 not found: ID does not exist"
Sep 12 00:40:52.553246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.553225    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:40:52.557009 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.556974    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-39d42b/test-model-39d42b-kserve-f6f8bdfdf-mqzs7"]
Sep 12 00:40:52.871940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.871901    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:40:52.879326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:52.879290    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:40:53.228591 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:53.228552    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 12 00:40:53.228919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:53.228889    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 12 00:40:54.227986 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:54.227947    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 12 00:40:54.228602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:54.228568    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 12 00:40:54.343261 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:54.343210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:54.375289 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:54.375250    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b35d18-1403-49bb-ba94-e101d5e60333" path="/var/lib/kubelet/pods/55b35d18-1403-49bb-ba94-e101d5e60333/volumes"
Sep 12 00:40:55.228299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:55.228254    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 12 00:40:55.228911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:55.228878    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 12 00:40:56.232618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:56.232579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:40:56.233814 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:40:56.233792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:41:00.713954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:00.713921    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:41:00.714371 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:00.714184    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy" containerID="cri-o://1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58" gracePeriod=30
Sep 12 00:41:04.511464 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:04.511425    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"]
Sep 12 00:41:04.513894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:04.513870    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3c466c-jct47"]
Sep 12 00:41:05.982348 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:05.982289    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:41:06.100284 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100246    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100344    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100532    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100557    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:06.100747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100612    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48pqv\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv\") pod \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\" (UID: \"8095d9fc-9e6b-4073-aa70-a6787fea3af5\") "
Sep 12 00:41:06.100747 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100679    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data" (OuterVolumeSpecName: "istio-data") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.100898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.101028    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.101049    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.101064    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.101075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.101078    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.101387 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.101202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:06.102706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.102681    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:41:06.102848 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.102729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token" (OuterVolumeSpecName: "istio-token") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:06.102848 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.102796    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:06.102951 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.102860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv" (OuterVolumeSpecName: "kube-api-access-48pqv") pod "8095d9fc-9e6b-4073-aa70-a6787fea3af5" (UID: "8095d9fc-9e6b-4073-aa70-a6787fea3af5"). InnerVolumeSpecName "kube-api-access-48pqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:06.202125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.202086    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.202125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.202118    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.202125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.202134    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.202384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.202146    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48pqv\" (UniqueName: \"kubernetes.io/projected/8095d9fc-9e6b-4073-aa70-a6787fea3af5-kube-api-access-48pqv\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.202384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.202157    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8095d9fc-9e6b-4073-aa70-a6787fea3af5-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:06.375150 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.375073    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c2059d-c9c1-425c-a037-85e8e05da7cf" path="/var/lib/kubelet/pods/e9c2059d-c9c1-425c-a037-85e8e05da7cf/volumes"
Sep 12 00:41:06.621619 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.621586    2580 generic.go:356] "Generic (PLEG): container finished" podID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerID="1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58" exitCode=0
Sep 12 00:41:06.621860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.621626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerDied","Data":"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"}
Sep 12 00:41:06.621860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.621647    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"
Sep 12 00:41:06.621860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.621674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8" event={"ID":"8095d9fc-9e6b-4073-aa70-a6787fea3af5","Type":"ContainerDied","Data":"5e979a460bed23d13168bd21f39784268838987043207fa274f194461f1991fc"}
Sep 12 00:41:06.621860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.621698    2580 scope.go:117] "RemoveContainer" containerID="1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"
Sep 12 00:41:06.631198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.631174    2580 scope.go:117] "RemoveContainer" containerID="1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"
Sep 12 00:41:06.637937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.637911    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:41:06.640849 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.640825    2580 scope.go:117] "RemoveContainer" containerID="1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"
Sep 12 00:41:06.641173 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:06.641150    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58\": container with ID starting with 1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58 not found: ID does not exist" containerID="1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"
Sep 12 00:41:06.641260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.641181    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58"} err="failed to get container status \"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58\": rpc error: code = NotFound desc = could not find container \"1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58\": container with ID starting with 1408a8f24391bdebbf181ce5cc132d188202a15dcab718d8fb38cea853efcf58 not found: ID does not exist"
Sep 12 00:41:06.641260 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.641201    2580 scope.go:117] "RemoveContainer" containerID="1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"
Sep 12 00:41:06.641466 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:06.641448    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20\": container with ID starting with 1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20 not found: ID does not exist" containerID="1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"
Sep 12 00:41:06.641536 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.641472    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20"} err="failed to get container status \"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20\": rpc error: code = NotFound desc = could not find container \"1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20\": container with ID starting with 1f5f683eb842a32c4236a341f0753ce155e2d55856b101e3a54f022d4acf7f20 not found: ID does not exist"
Sep 12 00:41:06.641899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:06.641882    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-39d42b-openshift-default-5658d889b8-kw9n8"]
Sep 12 00:41:07.506495 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:07.506456    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:41:07.892098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:07.892007    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" podUID="24119ca5-6c0c-4aed-a0af-9a364149b5de" containerName="main" containerID="cri-o://28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8" gracePeriod=45
Sep 12 00:41:07.902198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:07.902139    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" podUID="c0f8571f-4feb-4d3b-a175-a16da7c63434" containerName="main" containerID="cri-o://919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2" gracePeriod=45
Sep 12 00:41:08.173805 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.173756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:41:08.176374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.176354    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:41:08.323228 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.323399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6z6d\" (UniqueName: \"kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.323399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323301    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.323399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323325    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.323399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdcmf\" (UniqueName: \"kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323580    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.323643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache\") pod \"24119ca5-6c0c-4aed-a0af-9a364149b5de\" (UID: \"24119ca5-6c0c-4aed-a0af-9a364149b5de\") "
Sep 12 00:41:08.324045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323648    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm\") pod \"c0f8571f-4feb-4d3b-a175-a16da7c63434\" (UID: \"c0f8571f-4feb-4d3b-a175-a16da7c63434\") "
Sep 12 00:41:08.324045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home" (OuterVolumeSpecName: "home") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.324045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.323981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home" (OuterVolumeSpecName: "home") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.324213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324096    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.324213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324117    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.324213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.324378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache" (OuterVolumeSpecName: "model-cache") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.324378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324296    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.324378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.324317    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.326850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.326523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm" (OuterVolumeSpecName: "dshm") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.326850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.326694    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf" (OuterVolumeSpecName: "kube-api-access-mdcmf") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "kube-api-access-mdcmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:08.326850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.326838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d" (OuterVolumeSpecName: "kube-api-access-t6z6d") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "kube-api-access-t6z6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:08.326850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.326848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:08.326850 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.327094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm" (OuterVolumeSpecName: "dshm") pod "c0f8571f-4feb-4d3b-a175-a16da7c63434" (UID: "c0f8571f-4feb-4d3b-a175-a16da7c63434"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:08.327746 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.327571    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24119ca5-6c0c-4aed-a0af-9a364149b5de" (UID: "24119ca5-6c0c-4aed-a0af-9a364149b5de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:08.374723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.374688    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" path="/var/lib/kubelet/pods/8095d9fc-9e6b-4073-aa70-a6787fea3af5/volumes"
Sep 12 00:41:08.425053 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425021    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6z6d\" (UniqueName: \"kubernetes.io/projected/c0f8571f-4feb-4d3b-a175-a16da7c63434-kube-api-access-t6z6d\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425053 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425052    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8571f-4feb-4d3b-a175-a16da7c63434-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425063    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425072    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425081    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdcmf\" (UniqueName: \"kubernetes.io/projected/24119ca5-6c0c-4aed-a0af-9a364149b5de-kube-api-access-mdcmf\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425089    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24119ca5-6c0c-4aed-a0af-9a364149b5de-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425097    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425107    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24119ca5-6c0c-4aed-a0af-9a364149b5de-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.425254 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.425114    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8571f-4feb-4d3b-a175-a16da7c63434-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:08.635719 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.635689    2580 generic.go:356] "Generic (PLEG): container finished" podID="24119ca5-6c0c-4aed-a0af-9a364149b5de" containerID="28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8" exitCode=0
Sep 12 00:41:08.636200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.635739    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"
Sep 12 00:41:08.636200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.635796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" event={"ID":"24119ca5-6c0c-4aed-a0af-9a364149b5de","Type":"ContainerDied","Data":"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"}
Sep 12 00:41:08.636200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.635836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg" event={"ID":"24119ca5-6c0c-4aed-a0af-9a364149b5de","Type":"ContainerDied","Data":"907c1b575b67a84c62a0246c2b34f74b84ed29f8bddd5ba6468989e46107e82b"}
Sep 12 00:41:08.636200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.635858    2580 scope.go:117] "RemoveContainer" containerID="28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"
Sep 12 00:41:08.637403 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.637378    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0f8571f-4feb-4d3b-a175-a16da7c63434" containerID="919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2" exitCode=0
Sep 12 00:41:08.637513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.637435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"
Sep 12 00:41:08.637513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.637444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" event={"ID":"c0f8571f-4feb-4d3b-a175-a16da7c63434","Type":"ContainerDied","Data":"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"}
Sep 12 00:41:08.637513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.637483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf" event={"ID":"c0f8571f-4feb-4d3b-a175-a16da7c63434","Type":"ContainerDied","Data":"c9da24c985e053f856c5de797e4e366d27830fa19a573d0545b0008ab96f31f2"}
Sep 12 00:41:08.645082 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.645061    2580 scope.go:117] "RemoveContainer" containerID="28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"
Sep 12 00:41:08.645345 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:08.645324    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8\": container with ID starting with 28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8 not found: ID does not exist" containerID="28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"
Sep 12 00:41:08.645419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.645350    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8"} err="failed to get container status \"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8\": rpc error: code = NotFound desc = could not find container \"28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8\": container with ID starting with 28f8682600013b522ca76ef0801dd55916fba53a8e1784270fcc0686cbd379f8 not found: ID does not exist"
Sep 12 00:41:08.645419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.645366    2580 scope.go:117] "RemoveContainer" containerID="919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"
Sep 12 00:41:08.653673 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.653649    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:41:08.654045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.654029    2580 scope.go:117] "RemoveContainer" containerID="919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"
Sep 12 00:41:08.654319 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:08.654302    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2\": container with ID starting with 919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2 not found: ID does not exist" containerID="919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"
Sep 12 00:41:08.654381 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.654327    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2"} err="failed to get container status \"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2\": rpc error: code = NotFound desc = could not find container \"919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2\": container with ID starting with 919af19c569cfbeb4d884991888fac74cbcee15748a717a2617d8c328d4c2cf2 not found: ID does not exist"
Sep 12 00:41:08.659174 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.659153    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/distinct-w0-3c466c-kserve-6fc8fb45c6-6p5hg"]
Sep 12 00:41:08.668199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.668177    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:41:08.673447 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.673423    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-3c466c/premium-w0-3c466c-kserve-c9dcc6f9c-6dqnf"]
Sep 12 00:41:08.939988 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.939955    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:41:08.940335 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940322    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b35d18-1403-49bb-ba94-e101d5e60333" containerName="main"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940336    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b35d18-1403-49bb-ba94-e101d5e60333" containerName="main"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940355    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940360    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940367    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940372    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940378    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24119ca5-6c0c-4aed-a0af-9a364149b5de" containerName="main"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940383    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24119ca5-6c0c-4aed-a0af-9a364149b5de" containerName="main"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940389    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c2059d-c9c1-425c-a037-85e8e05da7cf" containerName="revoke-keys"
Sep 12 00:41:08.940391 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940394    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c2059d-c9c1-425c-a037-85e8e05da7cf" containerName="revoke-keys"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940407    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f8571f-4feb-4d3b-a175-a16da7c63434" containerName="main"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940412    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f8571f-4feb-4d3b-a175-a16da7c63434" containerName="main"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940461    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b35d18-1403-49bb-ba94-e101d5e60333" containerName="main"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940470    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940477    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f8571f-4feb-4d3b-a175-a16da7c63434" containerName="main"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940483    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8095d9fc-9e6b-4073-aa70-a6787fea3af5" containerName="istio-proxy"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940489    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24119ca5-6c0c-4aed-a0af-9a364149b5de" containerName="main"
Sep 12 00:41:08.940658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.940496    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c2059d-c9c1-425c-a037-85e8e05da7cf" containerName="revoke-keys"
Sep 12 00:41:08.943414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.943394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:08.945628 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.945591    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-e88680\""
Sep 12 00:41:08.945705 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.945635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-e88680-dockercfg-sz7cv\""
Sep 12 00:41:08.953220 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.953200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:41:08.969895 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.969870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:41:08.972118 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.972102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:08.983805 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:08.983740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:41:09.031147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.031121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.031281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.031197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzlwx\" (UniqueName: \"kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.031281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.031226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zwnk\" (UniqueName: \"kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.031281 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.031247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.132581 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.132537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.132795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.132646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzlwx\" (UniqueName: \"kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.132795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.132676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zwnk\" (UniqueName: \"kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.132795 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.132713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.133336 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.133316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.133498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.133468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.141016 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.140992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzlwx\" (UniqueName: \"kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx\") pod \"payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.141136 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.141110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zwnk\" (UniqueName: \"kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk\") pod \"payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.253994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.253909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.283237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.283205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.528827 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.528801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:41:09.530029 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:09.530002    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecab940e_efb8_4e5f_beda_c676cb5e3fc4.slice/crio-216ee919b50cdcf761ab0d4cc749a3eb5ce0fecafd66766274b80be88e912dc6 WatchSource:0}: Error finding container 216ee919b50cdcf761ab0d4cc749a3eb5ce0fecafd66766274b80be88e912dc6: Status 404 returned error can't find the container with id 216ee919b50cdcf761ab0d4cc749a3eb5ce0fecafd66766274b80be88e912dc6
Sep 12 00:41:09.568969 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.568942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:41:09.577956 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:09.577930    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251cacb4_c30f_4eed_85f3_c656d5de3271.slice/crio-08616722bd884d9d3a9ff9b5f14916dc9c7063238a2a1e8a6a1ff3a95fa1f0a4 WatchSource:0}: Error finding container 08616722bd884d9d3a9ff9b5f14916dc9c7063238a2a1e8a6a1ff3a95fa1f0a4: Status 404 returned error can't find the container with id 08616722bd884d9d3a9ff9b5f14916dc9c7063238a2a1e8a6a1ff3a95fa1f0a4
Sep 12 00:41:09.655210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.655175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" event={"ID":"ecab940e-efb8-4e5f-beda-c676cb5e3fc4","Type":"ContainerStarted","Data":"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"}
Sep 12 00:41:09.655668 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.655265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" event={"ID":"ecab940e-efb8-4e5f-beda-c676cb5e3fc4","Type":"ContainerStarted","Data":"216ee919b50cdcf761ab0d4cc749a3eb5ce0fecafd66766274b80be88e912dc6"}
Sep 12 00:41:09.655668 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.655297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:09.657429 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.657397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" event={"ID":"251cacb4-c30f-4eed-85f3-c656d5de3271","Type":"ContainerStarted","Data":"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"}
Sep 12 00:41:09.657586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.657453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" event={"ID":"251cacb4-c30f-4eed-85f3-c656d5de3271","Type":"ContainerStarted","Data":"08616722bd884d9d3a9ff9b5f14916dc9c7063238a2a1e8a6a1ff3a95fa1f0a4"}
Sep 12 00:41:09.657735 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.657705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:09.670160 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.670108    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" podStartSLOduration=1.670089277 podStartE2EDuration="1.670089277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:09.669154728 +0000 UTC m=+1465.803144949" watchObservedRunningTime="2026-09-12 00:41:09.670089277 +0000 UTC m=+1465.804079503"
Sep 12 00:41:09.688038 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:09.687984    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" podStartSLOduration=1.687966769 podStartE2EDuration="1.687966769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:09.685292662 +0000 UTC m=+1465.819282888" watchObservedRunningTime="2026-09-12 00:41:09.687966769 +0000 UTC m=+1465.821956996"
Sep 12 00:41:10.374031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:10.373995    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24119ca5-6c0c-4aed-a0af-9a364149b5de" path="/var/lib/kubelet/pods/24119ca5-6c0c-4aed-a0af-9a364149b5de/volumes"
Sep 12 00:41:10.374400 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:10.374388    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f8571f-4feb-4d3b-a175-a16da7c63434" path="/var/lib/kubelet/pods/c0f8571f-4feb-4d3b-a175-a16da7c63434/volumes"
Sep 12 00:41:11.083713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:11.083683    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:41:11.084103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:11.083899    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy" containerID="cri-o://bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328" gracePeriod=30
Sep 12 00:41:16.340482 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.340453    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:41:16.512519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512598    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th7mc\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512760    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.512912 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.513075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.513075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket\") pod \"9271239b-19a7-4dc2-8134-2eecc27bc446\" (UID: \"9271239b-19a7-4dc2-8134-2eecc27bc446\") "
Sep 12 00:41:16.513075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.512968    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:16.513075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513033    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data" (OuterVolumeSpecName: "istio-data") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:16.513330 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513311    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:16.513437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:16.513523 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513505    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9271239b-19a7-4dc2-8134-2eecc27bc446-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.513585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513525    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.513585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513540    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.513585 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513555    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.513805 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.513757    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:16.514880 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.514853    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:16.515097 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.515072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token" (OuterVolumeSpecName: "istio-token") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:16.515207 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.515185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc" (OuterVolumeSpecName: "kube-api-access-th7mc") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "kube-api-access-th7mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:16.515278 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.515188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9271239b-19a7-4dc2-8134-2eecc27bc446" (UID: "9271239b-19a7-4dc2-8134-2eecc27bc446"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:41:16.614263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.614223    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th7mc\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-kube-api-access-th7mc\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.614263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.614258    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.614263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.614270    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.614507 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.614280    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.614507 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.614293    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9271239b-19a7-4dc2-8134-2eecc27bc446-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:16.702776 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.702726    2580 generic.go:356] "Generic (PLEG): container finished" podID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerID="bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328" exitCode=0
Sep 12 00:41:16.702943 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.702787    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"
Sep 12 00:41:16.702943 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.702814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerDied","Data":"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"}
Sep 12 00:41:16.702943 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.702869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz" event={"ID":"9271239b-19a7-4dc2-8134-2eecc27bc446","Type":"ContainerDied","Data":"d8a5e08e82096d4c9f0cc386c4b6c25857f8a875bfb10f1fb24b82054dee6ed6"}
Sep 12 00:41:16.702943 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.702891    2580 scope.go:117] "RemoveContainer" containerID="bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"
Sep 12 00:41:16.712530 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.712513    2580 scope.go:117] "RemoveContainer" containerID="d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"
Sep 12 00:41:16.720791 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.720773    2580 scope.go:117] "RemoveContainer" containerID="bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"
Sep 12 00:41:16.721088 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:16.721055    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328\": container with ID starting with bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328 not found: ID does not exist" containerID="bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"
Sep 12 00:41:16.721199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.721097    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328"} err="failed to get container status \"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328\": rpc error: code = NotFound desc = could not find container \"bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328\": container with ID starting with bfd9409d9fbb70b582cb1717db9e85e5af26b759131834ef159ecd01f0390328 not found: ID does not exist"
Sep 12 00:41:16.721199 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.721121    2580 scope.go:117] "RemoveContainer" containerID="d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"
Sep 12 00:41:16.722375 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:16.722345    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988\": container with ID starting with d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988 not found: ID does not exist" containerID="d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"
Sep 12 00:41:16.722472 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.722381    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988"} err="failed to get container status \"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988\": rpc error: code = NotFound desc = could not find container \"d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988\": container with ID starting with d91cb2023055d7ee977326e4ffd25e462cc5d5cfde45f2bcce50908b1d672988 not found: ID does not exist"
Sep 12 00:41:16.724689 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.724665    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:41:16.730397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:16.730378    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c466c-openshift-default-74b6fd8454-xx9jz"]
Sep 12 00:41:18.375351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:18.375316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" path="/var/lib/kubelet/pods/9271239b-19a7-4dc2-8134-2eecc27bc446/volumes"
Sep 12 00:41:20.668408 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:20.668379    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:41:20.668800 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:20.668456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:41:22.526534 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.526467    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" podUID="fca3fd9c-c36f-4885-a456-3186d904e25d" containerName="main" containerID="cri-o://39ef3d20b87d73a33cd564cdd31d7b6ab38a7ab0d6aa2b846ef1a00fd47dbd89" gracePeriod=45
Sep 12 00:41:22.738750 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.738720    2580 generic.go:356] "Generic (PLEG): container finished" podID="fca3fd9c-c36f-4885-a456-3186d904e25d" containerID="39ef3d20b87d73a33cd564cdd31d7b6ab38a7ab0d6aa2b846ef1a00fd47dbd89" exitCode=0
Sep 12 00:41:22.738921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.738791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" event={"ID":"fca3fd9c-c36f-4885-a456-3186d904e25d","Type":"ContainerDied","Data":"39ef3d20b87d73a33cd564cdd31d7b6ab38a7ab0d6aa2b846ef1a00fd47dbd89"}
Sep 12 00:41:22.776574 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.776520    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:41:22.874402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874363    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64dcf\" (UniqueName: \"kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874656 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm\") pod \"fca3fd9c-c36f-4885-a456-3186d904e25d\" (UID: \"fca3fd9c-c36f-4885-a456-3186d904e25d\") "
Sep 12 00:41:22.874890 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache" (OuterVolumeSpecName: "model-cache") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:22.874890 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874716    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home" (OuterVolumeSpecName: "home") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:22.874890 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874866    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:22.875018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874940    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:22.875018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.874959    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:22.876848 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.876820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf" (OuterVolumeSpecName: "kube-api-access-64dcf") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "kube-api-access-64dcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:22.876942 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.876841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm" (OuterVolumeSpecName: "dshm") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:41:22.876942 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.876900    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fca3fd9c-c36f-4885-a456-3186d904e25d" (UID: "fca3fd9c-c36f-4885-a456-3186d904e25d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:22.975402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.975369    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64dcf\" (UniqueName: \"kubernetes.io/projected/fca3fd9c-c36f-4885-a456-3186d904e25d-kube-api-access-64dcf\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:22.975402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.975395    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:22.975402 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.975406    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fca3fd9c-c36f-4885-a456-3186d904e25d-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:22.975610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:22.975414    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca3fd9c-c36f-4885-a456-3186d904e25d-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:23.745863 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:23.745831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"
Sep 12 00:41:23.746290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:23.745865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5" event={"ID":"fca3fd9c-c36f-4885-a456-3186d904e25d","Type":"ContainerDied","Data":"b589d5ab3335e60f711ef50df64cfe81b3a7626995b112d8610bffae0d9881e0"}
Sep 12 00:41:23.746290 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:23.745918    2580 scope.go:117] "RemoveContainer" containerID="39ef3d20b87d73a33cd564cdd31d7b6ab38a7ab0d6aa2b846ef1a00fd47dbd89"
Sep 12 00:41:23.766676 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:23.766640    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:41:23.770035 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:23.770012    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-139288/e2e-conflict-b-40370e1e-kserve-6b58f97794-zpnz5"]
Sep 12 00:41:24.375212 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:24.375178    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca3fd9c-c36f-4885-a456-3186d904e25d" path="/var/lib/kubelet/pods/fca3fd9c-c36f-4885-a456-3186d904e25d/volumes"
Sep 12 00:41:35.795424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795384    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:41:35.795822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795807    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.795894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795824    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.795894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795835    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca3fd9c-c36f-4885-a456-3186d904e25d" containerName="main"
Sep 12 00:41:35.795894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795840    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca3fd9c-c36f-4885-a456-3186d904e25d" containerName="main"
Sep 12 00:41:35.795894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795850    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.795894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795856    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.796133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795921    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.796133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795931    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9271239b-19a7-4dc2-8134-2eecc27bc446" containerName="istio-proxy"
Sep 12 00:41:35.796133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.795939    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca3fd9c-c36f-4885-a456-3186d904e25d" containerName="main"
Sep 12 00:41:35.800301 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.800281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.802488 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.802467    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-e37bfb-openshift-default-dockercfg-b6v6s\""
Sep 12 00:41:35.811197 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.811170    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:41:35.896278 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896337    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896432 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph8v5\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.896624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.896576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998034 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.997998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph8v5\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998211 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998718 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998718 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.998845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.998715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:35.999141 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:35.999108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.000498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.000473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.000685 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.000666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.005504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.005472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.005660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.005637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph8v5\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5\") pod \"e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") " pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.112556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.112473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:36.387433 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.387411    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:41:36.389637 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:36.389607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099e16c8_2879_4b8c_a3a6_31e58f6cad28.slice/crio-811d351f82b46f34ee1f9279b20e130f5c2ebc0be6d41e5387101e535e889611 WatchSource:0}: Error finding container 811d351f82b46f34ee1f9279b20e130f5c2ebc0be6d41e5387101e535e889611: Status 404 returned error can't find the container with id 811d351f82b46f34ee1f9279b20e130f5c2ebc0be6d41e5387101e535e889611
Sep 12 00:41:36.392487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.392450    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:36.392603 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.392536    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:36.392603 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.392583    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:36.823961 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.823907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerStarted","Data":"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"}
Sep 12 00:41:36.824367 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.824014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerStarted","Data":"811d351f82b46f34ee1f9279b20e130f5c2ebc0be6d41e5387101e535e889611"}
Sep 12 00:41:36.843255 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:36.843206    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podStartSLOduration=1.843192965 podStartE2EDuration="1.843192965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:36.841602033 +0000 UTC m=+1492.975592250" watchObservedRunningTime="2026-09-12 00:41:36.843192965 +0000 UTC m=+1492.977183189"
Sep 12 00:41:37.112884 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:37.112761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:37.118023 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:37.117994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:37.830061 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:37.830022    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:37.830839 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:37.830820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:41:40.450690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.450653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"]
Sep 12 00:41:40.454166 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.454149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:40.456331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.456305    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:41:40.463191 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.463167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"]
Sep 12 00:41:40.543802 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.543748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd\" (UID: \"7d22843e-f571-45b3-8eb4-c2e6d07bee8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:40.645081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.645041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd\" (UID: \"7d22843e-f571-45b3-8eb4-c2e6d07bee8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:40.646100 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.645686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd\" (UID: \"7d22843e-f571-45b3-8eb4-c2e6d07bee8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:40.767276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:40.767196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:41.037313 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:41.037282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"]
Sep 12 00:41:41.039870 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:41.039840    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d22843e_f571_45b3_8eb4_c2e6d07bee8e.slice/crio-7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754 WatchSource:0}: Error finding container 7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754: Status 404 returned error can't find the container with id 7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754
Sep 12 00:41:41.853643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:41.853603    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d22843e-f571-45b3-8eb4-c2e6d07bee8e" containerID="2979b7e6879883b75199748e4e24d5f0616799e35b26dcb87ede1dd15d453b18" exitCode=0
Sep 12 00:41:41.854077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:41.853685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd" event={"ID":"7d22843e-f571-45b3-8eb4-c2e6d07bee8e","Type":"ContainerDied","Data":"2979b7e6879883b75199748e4e24d5f0616799e35b26dcb87ede1dd15d453b18"}
Sep 12 00:41:41.854077 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:41.853749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd" event={"ID":"7d22843e-f571-45b3-8eb4-c2e6d07bee8e","Type":"ContainerStarted","Data":"7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754"}
Sep 12 00:41:42.617174 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.617134    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:41:42.621352 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.621329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.623447 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.623425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-e37bfb-serving-cert\""
Sep 12 00:41:42.628964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.628941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:41:42.664813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.664787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv9md\" (UniqueName: \"kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.664940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.664832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.664940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.664881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.664940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.664907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.765684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.765651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv9md\" (UniqueName: \"kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.765903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.765693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.765903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.765734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.765903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.765758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.765903 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:42.765864    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-serving-cert: secret "maas-api-e2e-shared-b-w3-e37bfb-serving-cert" not found
Sep 12 00:41:42.766144 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:41:42.765926    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls podName:139c5f59-0f7d-4615-b2ce-2462c42d2dac nodeName:}" failed. No retries permitted until 2026-09-12 00:41:43.265910155 +0000 UTC m=+1499.399900358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls") pod "maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" (UID: "139c5f59-0f7d-4615-b2ce-2462c42d2dac") : secret "maas-api-e2e-shared-b-w3-e37bfb-serving-cert" not found
Sep 12 00:41:42.766387 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.766369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.768198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.768175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.773534 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.773511    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv9md\" (UniqueName: \"kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:42.991002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:42.990977    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:43.068798 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.068752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca\") pod \"7d22843e-f571-45b3-8eb4-c2e6d07bee8e\" (UID: \"7d22843e-f571-45b3-8eb4-c2e6d07bee8e\") "
Sep 12 00:41:43.069142 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.069122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d22843e-f571-45b3-8eb4-c2e6d07bee8e" (UID: "7d22843e-f571-45b3-8eb4-c2e6d07bee8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:43.170124 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.170094    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d22843e-f571-45b3-8eb4-c2e6d07bee8e-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:41:43.270927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.270897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:43.273334 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.273300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:43.535167 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.535052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:43.808288 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.808259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:41:43.810153 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:43.810126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139c5f59_0f7d_4615_b2ce_2462c42d2dac.slice/crio-2d40d74068b9aeb232321eddff10ab35f54d55af4ae222cb2396f34d68eb3827 WatchSource:0}: Error finding container 2d40d74068b9aeb232321eddff10ab35f54d55af4ae222cb2396f34d68eb3827: Status 404 returned error can't find the container with id 2d40d74068b9aeb232321eddff10ab35f54d55af4ae222cb2396f34d68eb3827
Sep 12 00:41:43.812001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.811981    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:41:43.867452 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.867426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"
Sep 12 00:41:43.867452 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.867444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd" event={"ID":"7d22843e-f571-45b3-8eb4-c2e6d07bee8e","Type":"ContainerDied","Data":"7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754"}
Sep 12 00:41:43.867655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.867486    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f658f7198b3fbbe202c27fbbbf3b1cff7faecee270f3c35be2520a567f37754"
Sep 12 00:41:43.868821 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:43.868794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" event={"ID":"139c5f59-0f7d-4615-b2ce-2462c42d2dac","Type":"ContainerStarted","Data":"2d40d74068b9aeb232321eddff10ab35f54d55af4ae222cb2396f34d68eb3827"}
Sep 12 00:41:44.876339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:44.876235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" event={"ID":"139c5f59-0f7d-4615-b2ce-2462c42d2dac","Type":"ContainerStarted","Data":"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"}
Sep 12 00:41:44.876339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:44.876327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:44.898419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:44.898366    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" podStartSLOduration=2.537943293 podStartE2EDuration="2.898351516s" totalImagesPullingTime="360.408223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:44.896882047 +0000 UTC m=+1501.030872284" watchObservedRunningTime="2026-09-12 00:41:44.898351516 +0000 UTC m=+1501.032341740"
Sep 12 00:41:47.999777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:47.999728    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:41:48.000156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.000114    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d22843e-f571-45b3-8eb4-c2e6d07bee8e" containerName="revoke-keys"
Sep 12 00:41:48.000156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.000125    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d22843e-f571-45b3-8eb4-c2e6d07bee8e" containerName="revoke-keys"
Sep 12 00:41:48.000231 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.000196    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d22843e-f571-45b3-8eb4-c2e6d07bee8e" containerName="revoke-keys"
Sep 12 00:41:48.004741 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.004720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.006998 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.006978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"distinct-w4-139288-kserve-self-signed-certs\""
Sep 12 00:41:48.014348 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.014325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:41:48.115853 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.115813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.116027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.115861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.116027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.115920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.116027 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.116003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpp5l\" (UniqueName: \"kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.116182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.116113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.116182 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.116141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.217891 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.217759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpp5l\" (UniqueName: \"kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.218088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.218055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.218088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.218066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.218200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.218121    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.219921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.219893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.220331 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.220309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.225229 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.225206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpp5l\" (UniqueName: \"kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l\") pod \"distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") " pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.315142 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.315061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:48.585903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.585870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:41:48.587145 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:41:48.587116    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0eb7ce6_1271_4e61_a562_44a0353d9def.slice/crio-b1cd5fea0fdc6d98dd4ca071a7fdb0e03d5e67fd9eb304f0b0bfe5749bfd2b57 WatchSource:0}: Error finding container b1cd5fea0fdc6d98dd4ca071a7fdb0e03d5e67fd9eb304f0b0bfe5749bfd2b57: Status 404 returned error can't find the container with id b1cd5fea0fdc6d98dd4ca071a7fdb0e03d5e67fd9eb304f0b0bfe5749bfd2b57
Sep 12 00:41:48.904483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.904385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" event={"ID":"d0eb7ce6-1271-4e61-a562-44a0353d9def","Type":"ContainerStarted","Data":"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"}
Sep 12 00:41:48.904483 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.904431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" event={"ID":"d0eb7ce6-1271-4e61-a562-44a0353d9def","Type":"ContainerStarted","Data":"b1cd5fea0fdc6d98dd4ca071a7fdb0e03d5e67fd9eb304f0b0bfe5749bfd2b57"}
Sep 12 00:41:48.922511 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:48.922460    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" podStartSLOduration=1.922444375 podStartE2EDuration="1.922444375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:48.919809771 +0000 UTC m=+1505.053799997" watchObservedRunningTime="2026-09-12 00:41:48.922444375 +0000 UTC m=+1505.056434600"
Sep 12 00:41:50.889384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:50.889353    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:41:56.347263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:56.347220    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"]
Sep 12 00:41:56.352492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:56.352456    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cfdf5d6c-b4gdd"]
Sep 12 00:41:56.375088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:56.375051    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d22843e-f571-45b3-8eb4-c2e6d07bee8e" path="/var/lib/kubelet/pods/7d22843e-f571-45b3-8eb4-c2e6d07bee8e/volumes"
Sep 12 00:41:58.315707 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.315666    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:58.315707 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.315715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:58.328611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.328582    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:58.982553 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.982521    2580 generic.go:356] "Generic (PLEG): container finished" podID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerID="d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d" exitCode=0
Sep 12 00:41:58.982755 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.982594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerDied","Data":"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"}
Sep 12 00:41:58.983065 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.983048    2580 scope.go:117] "RemoveContainer" containerID="d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"
Sep 12 00:41:58.984238 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.984208    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:58.984322 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.984274    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:58.984322 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.984304    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 12 00:41:58.994323 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:58.994297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:41:59.991319 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:41:59.991286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerStarted","Data":"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"}
Sep 12 00:42:00.112925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:00.112890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:42:00.113205 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:00.113181    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 12 00:42:00.113268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:00.113247    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 12 00:42:01.113898 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:01.113858    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 12 00:42:01.114382 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:01.113924    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 12 00:42:02.113220 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:02.113180    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 12 00:42:02.113414 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:02.113242    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 12 00:42:03.113217 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:03.113170    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 12 00:42:03.113631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:03.113233    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 12 00:42:04.113184 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:04.113150    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 12 00:42:04.113368 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:04.113213    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 12 00:42:05.118090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:05.118060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:42:05.118475 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:05.118455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:42:05.119251 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:05.119236    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:42:12.717572 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.717465    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:42:12.720782 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.720744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.722868 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.722840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-139288\"/\"distinct-2-w4-139288-kserve-self-signed-certs\""
Sep 12 00:42:12.729603 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.729582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:42:12.771215 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.771215 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6j8\" (UniqueName: \"kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.771419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.771419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.771419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.771419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.771360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872226 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6j8\" (UniqueName: \"kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872489 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.872725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.872706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.874660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.874627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.874869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.874852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:12.879504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:12.879481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6j8\" (UniqueName: \"kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8\") pod \"distinct-2-w4-139288-kserve-57dd999bd7-256h7\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") " pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:13.031894 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:13.031802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:13.304351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:13.304316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:42:13.305262 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:42:13.305237    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b5ec12a_2718_47c1_bfb6_89e7ea157da1.slice/crio-e8a0cc7123b33e949a43bfafeb9d3305e037039e799996d7f9c969d961a98bd7 WatchSource:0}: Error finding container e8a0cc7123b33e949a43bfafeb9d3305e037039e799996d7f9c969d961a98bd7: Status 404 returned error can't find the container with id e8a0cc7123b33e949a43bfafeb9d3305e037039e799996d7f9c969d961a98bd7
Sep 12 00:42:14.074496 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:14.074452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" event={"ID":"2b5ec12a-2718-47c1-bfb6-89e7ea157da1","Type":"ContainerStarted","Data":"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"}
Sep 12 00:42:14.074926 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:14.074507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" event={"ID":"2b5ec12a-2718-47c1-bfb6-89e7ea157da1","Type":"ContainerStarted","Data":"e8a0cc7123b33e949a43bfafeb9d3305e037039e799996d7f9c969d961a98bd7"}
Sep 12 00:42:14.093994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:14.093941    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" podStartSLOduration=2.093923545 podStartE2EDuration="2.093923545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:42:14.090954601 +0000 UTC m=+1530.224944826" watchObservedRunningTime="2026-09-12 00:42:14.093923545 +0000 UTC m=+1530.227913798"
Sep 12 00:42:23.032111 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:23.032072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:23.032500 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:23.032128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:23.044537 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:23.044506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:23.143433 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:23.143403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:42:44.554114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:42:44.554081    2580 scope.go:117] "RemoveContainer" containerID="0213dad798844d17702efe3a9dc114b0afd19475a7aed250d8fadc45ab271f92"
Sep 12 00:43:10.208178 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:10.208135    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:43:10.208759 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:10.208387    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" containerID="cri-o://8788808e1885454f6a3b50083c6e5e0daca79d301d73ca81c691c097fb2ac136" gracePeriod=30
Sep 12 00:43:11.236060 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:11.236020    2580 patch_prober.go:28] interesting pod/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:43:11.236454 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:11.236099    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:43:15.430360 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.430326    2580 generic.go:356] "Generic (PLEG): container finished" podID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerID="8788808e1885454f6a3b50083c6e5e0daca79d301d73ca81c691c097fb2ac136" exitCode=0
Sep 12 00:43:15.430779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.430396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerDied","Data":"8788808e1885454f6a3b50083c6e5e0daca79d301d73ca81c691c097fb2ac136"}
Sep 12 00:43:15.430779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.430485    2580 scope.go:117] "RemoveContainer" containerID="8ba01883fdd11f2be621669db7b114b0d241203fc274bd1267f5c87ca76ba3fa"
Sep 12 00:43:15.469356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.469331    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:43:15.600434 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600344    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600434 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htfmd\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.600684 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data\") pod \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\" (UID: \"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd\") "
Sep 12 00:43:15.601082 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:43:15.601082 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.601047    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.601082 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.600935    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:43:15.601457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.601076    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data" (OuterVolumeSpecName: "istio-data") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:43:15.601457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.601108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:43:15.601457 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.601214    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:43:15.602845 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.602821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:43:15.603001 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.602977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd" (OuterVolumeSpecName: "kube-api-access-htfmd") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "kube-api-access-htfmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:43:15.603070 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.603013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:43:15.603070 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.603031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" (UID: "1aecb8de-0f4c-4d54-8668-0839e7dfbcbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:43:15.702103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702065    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702096    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702103 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702104    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702115    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702124    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702132    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htfmd\" (UniqueName: \"kubernetes.io/projected/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-kube-api-access-htfmd\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702140    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:15.702342 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:15.702150    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:43:16.438300 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:16.438263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8" event={"ID":"1aecb8de-0f4c-4d54-8668-0839e7dfbcbd","Type":"ContainerDied","Data":"77a2a2c18ab9a85581a93f3c8e031dac2c426d8312ead9c583d648b5f51c4175"}
Sep 12 00:43:16.438751 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:16.438328    2580 scope.go:117] "RemoveContainer" containerID="8788808e1885454f6a3b50083c6e5e0daca79d301d73ca81c691c097fb2ac136"
Sep 12 00:43:16.438751 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:16.438277    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"
Sep 12 00:43:16.455032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:16.455006    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:43:16.460657 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:16.460631    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cfdf5d6c-openshift-default-67c67769fb-pjmx8"]
Sep 12 00:43:18.374537 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:43:18.374501    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" path="/var/lib/kubelet/pods/1aecb8de-0f4c-4d54-8668-0839e7dfbcbd/volumes"
Sep 12 00:44:12.772328 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772295    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:44:12.772777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:12.772777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772688    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:12.772777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:12.772777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772705    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:12.772919 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.772789    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:12.775925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.775907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.778049 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.778026    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-829959\""
Sep 12 00:44:12.778142 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.778088    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-829959-dockercfg-d2fsr\""
Sep 12 00:44:12.786374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.786348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:44:12.865783 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.865726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqt5w\" (UniqueName: \"kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.865959 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.865869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.967008 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.966954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqt5w\" (UniqueName: \"kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.967223 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.967040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.967800 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.967742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:12.975811 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:12.975783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqt5w\" (UniqueName: \"kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w\") pod \"payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") " pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:13.087968 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.087878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:13.360378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.360348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:44:13.361757 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:44:13.361728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9710f59_139b_4720_8f2e_1c1fde01b94d.slice/crio-5c23d647dfabdf493d955004e8d8d21ab8cd651b560509e339fbbe23150521a5 WatchSource:0}: Error finding container 5c23d647dfabdf493d955004e8d8d21ab8cd651b560509e339fbbe23150521a5: Status 404 returned error can't find the container with id 5c23d647dfabdf493d955004e8d8d21ab8cd651b560509e339fbbe23150521a5
Sep 12 00:44:13.770055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.770014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" event={"ID":"d9710f59-139b-4720-8f2e-1c1fde01b94d","Type":"ContainerStarted","Data":"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"}
Sep 12 00:44:13.770263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.770091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" event={"ID":"d9710f59-139b-4720-8f2e-1c1fde01b94d","Type":"ContainerStarted","Data":"5c23d647dfabdf493d955004e8d8d21ab8cd651b560509e339fbbe23150521a5"}
Sep 12 00:44:13.770263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.770143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:13.786173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:13.786121    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" podStartSLOduration=1.7861061029999998 podStartE2EDuration="1.786106103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:13.78393164 +0000 UTC m=+1649.917921864" watchObservedRunningTime="2026-09-12 00:44:13.786106103 +0000 UTC m=+1649.920096353"
Sep 12 00:44:24.777567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:24.777508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:44:35.151119 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:35.151087    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:44:35.155635 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:35.155603    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:44:35.164558 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:35.164532    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:44:39.990700 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:39.990661    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:44:39.991165 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:39.991128    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aecb8de-0f4c-4d54-8668-0839e7dfbcbd" containerName="istio-proxy"
Sep 12 00:44:39.994194 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:39.994177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:39.996326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:39.996301    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-593845-dockercfg-kcxbl\""
Sep 12 00:44:39.996460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:39.996349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-593845\""
Sep 12 00:44:40.005267 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.005243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:44:40.034941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.034901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw8wd\" (UniqueName: \"kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.035106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.034950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.136343 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.136298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.136562 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.136426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw8wd\" (UniqueName: \"kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.137120 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.137094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.145234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.145211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw8wd\" (UniqueName: \"kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd\") pod \"payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") " pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.304722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.304641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.576455 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.576299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:44:40.579327 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:44:40.579296    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e3f18a_92d9_45fb_804e_a1c2637ff793.slice/crio-c545999c4c16b45ba4161a38f440998c541c12b9819f50f452ab401d56d76a99 WatchSource:0}: Error finding container c545999c4c16b45ba4161a38f440998c541c12b9819f50f452ab401d56d76a99: Status 404 returned error can't find the container with id c545999c4c16b45ba4161a38f440998c541c12b9819f50f452ab401d56d76a99
Sep 12 00:44:40.926253 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.926217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" event={"ID":"71e3f18a-92d9-45fb-804e-a1c2637ff793","Type":"ContainerStarted","Data":"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"}
Sep 12 00:44:40.926430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.926271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" event={"ID":"71e3f18a-92d9-45fb-804e-a1c2637ff793","Type":"ContainerStarted","Data":"c545999c4c16b45ba4161a38f440998c541c12b9819f50f452ab401d56d76a99"}
Sep 12 00:44:40.926430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.926324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:40.941276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:40.941213    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" podStartSLOduration=1.9411979050000001 podStartE2EDuration="1.941197905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:40.940023921 +0000 UTC m=+1677.074014148" watchObservedRunningTime="2026-09-12 00:44:40.941197905 +0000 UTC m=+1677.075188129"
Sep 12 00:44:44.591591 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:44.591557    2580 scope.go:117] "RemoveContainer" containerID="b5868752e57bf25aa5105fa0c464a7a05b6709e0085c2f428c0699fb6b773e41"
Sep 12 00:44:48.913096 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:48.913059    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:44:48.913549 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:48.913318    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" podUID="1ddcc8f9-93b0-4222-af14-062f437c2b21" containerName="maas-api" containerID="cri-o://65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f" gracePeriod=30
Sep 12 00:44:49.163422 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.163363    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:44:49.233075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.233037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhzpw\" (UniqueName: \"kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw\") pod \"1ddcc8f9-93b0-4222-af14-062f437c2b21\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") "
Sep 12 00:44:49.233256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.233090    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca\") pod \"1ddcc8f9-93b0-4222-af14-062f437c2b21\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") "
Sep 12 00:44:49.233256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.233118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs\") pod \"1ddcc8f9-93b0-4222-af14-062f437c2b21\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") "
Sep 12 00:44:49.233256 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.233223    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") pod \"1ddcc8f9-93b0-4222-af14-062f437c2b21\" (UID: \"1ddcc8f9-93b0-4222-af14-062f437c2b21\") "
Sep 12 00:44:49.233494 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.233469    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ddcc8f9-93b0-4222-af14-062f437c2b21" (UID: "1ddcc8f9-93b0-4222-af14-062f437c2b21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:44:49.235259 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.235234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw" (OuterVolumeSpecName: "kube-api-access-zhzpw") pod "1ddcc8f9-93b0-4222-af14-062f437c2b21" (UID: "1ddcc8f9-93b0-4222-af14-062f437c2b21"). InnerVolumeSpecName "kube-api-access-zhzpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:44:49.235339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.235278    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ddcc8f9-93b0-4222-af14-062f437c2b21" (UID: "1ddcc8f9-93b0-4222-af14-062f437c2b21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:44:49.235339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.235314    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1ddcc8f9-93b0-4222-af14-062f437c2b21" (UID: "1ddcc8f9-93b0-4222-af14-062f437c2b21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:44:49.334869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.334829    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-maas-api-tls\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:49.334869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.334864    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhzpw\" (UniqueName: \"kubernetes.io/projected/1ddcc8f9-93b0-4222-af14-062f437c2b21-kube-api-access-zhzpw\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:49.334869 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.334876    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddcc8f9-93b0-4222-af14-062f437c2b21-openshift-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:49.335187 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.334884    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddcc8f9-93b0-4222-af14-062f437c2b21-metrics-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:49.981098 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.981066    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ddcc8f9-93b0-4222-af14-062f437c2b21" containerID="65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f" exitCode=0
Sep 12 00:44:49.981540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.981108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"
Sep 12 00:44:49.981540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.981155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" event={"ID":"1ddcc8f9-93b0-4222-af14-062f437c2b21","Type":"ContainerDied","Data":"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"}
Sep 12 00:44:49.981540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.981208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx" event={"ID":"1ddcc8f9-93b0-4222-af14-062f437c2b21","Type":"ContainerDied","Data":"bb039291ee65e4a5f460b7af711e62aa209b4a87e98cb5747ee862bce51931ef"}
Sep 12 00:44:49.981540 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.981230    2580 scope.go:117] "RemoveContainer" containerID="65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"
Sep 12 00:44:49.991651 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.991632    2580 scope.go:117] "RemoveContainer" containerID="65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"
Sep 12 00:44:49.991940 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:44:49.991922    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f\": container with ID starting with 65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f not found: ID does not exist" containerID="65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"
Sep 12 00:44:49.992018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:49.991948    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f"} err="failed to get container status \"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f\": rpc error: code = NotFound desc = could not find container \"65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f\": container with ID starting with 65f8eb73f8ab75c1a9c82f5e11c227566f2d4c0b2f1e08a6381e447e409d6b7f not found: ID does not exist"
Sep 12 00:44:50.002551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.002529    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:44:50.004929 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.004910    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-139288-588856d668-2mtnx"]
Sep 12 00:44:50.171374 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.171326    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" podUID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" containerName="main" containerID="cri-o://068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5" gracePeriod=45
Sep 12 00:44:50.175125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.175092    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" podUID="d0eb7ce6-1271-4e61-a562-44a0353d9def" containerName="main" containerID="cri-o://62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725" gracePeriod=45
Sep 12 00:44:50.184173 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.184141    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" podUID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" containerName="main" containerID="cri-o://8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb" gracePeriod=45
Sep 12 00:44:50.376677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.376639    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ddcc8f9-93b0-4222-af14-062f437c2b21" path="/var/lib/kubelet/pods/1ddcc8f9-93b0-4222-af14-062f437c2b21/volumes"
Sep 12 00:44:50.436033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.436002    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:44:50.482985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.482955    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:44:50.484986 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.484965    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:44:50.546993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.546906    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.546993 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.546979    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.547225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.547225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547036    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.547225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.547399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547219    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.547399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547323    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home" (OuterVolumeSpecName: "home") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.547399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547334    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache" (OuterVolumeSpecName: "model-cache") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.547399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547425    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhg49\" (UniqueName: \"kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr6j8\" (UniqueName: \"kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.547631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir\") pod \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\" (UID: \"2b5ec12a-2718-47c1-bfb6-89e7ea157da1\") "
Sep 12 00:44:50.548050 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.548050 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547635    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.548050 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.548050 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.547708    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home" (OuterVolumeSpecName: "home") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.548273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.548115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.549867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir\") pod \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\" (UID: \"75b81ef8-fe97-48ef-b9c3-0e808f7d3809\") "
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.549916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.549967    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpp5l\" (UniqueName: \"kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.549994    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home\") pod \"d0eb7ce6-1271-4e61-a562-44a0353d9def\" (UID: \"d0eb7ce6-1271-4e61-a562-44a0353d9def\") "
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm" (OuterVolumeSpecName: "dshm") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.550419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550231    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550437    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550459    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550473    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550486    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550498    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550511    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550523    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550534    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550701    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.550861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.550823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm" (OuterVolumeSpecName: "dshm") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.551408 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:44:50.551408 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:44:50.551610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home" (OuterVolumeSpecName: "home") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.551694 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49" (OuterVolumeSpecName: "kube-api-access-rhg49") pod "75b81ef8-fe97-48ef-b9c3-0e808f7d3809" (UID: "75b81ef8-fe97-48ef-b9c3-0e808f7d3809"). InnerVolumeSpecName "kube-api-access-rhg49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:44:50.551694 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:44:50.551844 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.551800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8" (OuterVolumeSpecName: "kube-api-access-pr6j8") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "kube-api-access-pr6j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:44:50.552108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.552083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm" (OuterVolumeSpecName: "dshm") pod "2b5ec12a-2718-47c1-bfb6-89e7ea157da1" (UID: "2b5ec12a-2718-47c1-bfb6-89e7ea157da1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:44:50.552830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.552804    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l" (OuterVolumeSpecName: "kube-api-access-lpp5l") pod "d0eb7ce6-1271-4e61-a562-44a0353d9def" (UID: "d0eb7ce6-1271-4e61-a562-44a0353d9def"). InnerVolumeSpecName "kube-api-access-lpp5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:44:50.651324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651292    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpp5l\" (UniqueName: \"kubernetes.io/projected/d0eb7ce6-1271-4e61-a562-44a0353d9def-kube-api-access-lpp5l\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651318    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0eb7ce6-1271-4e61-a562-44a0353d9def-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651328    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651340    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhg49\" (UniqueName: \"kubernetes.io/projected/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-kube-api-access-rhg49\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651348    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr6j8\" (UniqueName: \"kubernetes.io/projected/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-kube-api-access-pr6j8\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651356    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ec12a-2718-47c1-bfb6-89e7ea157da1-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651366    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0eb7ce6-1271-4e61-a562-44a0353d9def-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651373    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651381    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.651561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.651388    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75b81ef8-fe97-48ef-b9c3-0e808f7d3809-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:44:50.996577 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.996542    2580 generic.go:356] "Generic (PLEG): container finished" podID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" containerID="8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb" exitCode=0
Sep 12 00:44:50.997094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.996598    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"
Sep 12 00:44:50.997094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.996628    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" event={"ID":"75b81ef8-fe97-48ef-b9c3-0e808f7d3809","Type":"ContainerDied","Data":"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"}
Sep 12 00:44:50.997094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.996685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz" event={"ID":"75b81ef8-fe97-48ef-b9c3-0e808f7d3809","Type":"ContainerDied","Data":"31784d7ac4d8706ee7972823a1016c7ad1847daf0d5964d51883b129a7ee4425"}
Sep 12 00:44:50.997094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.996709    2580 scope.go:117] "RemoveContainer" containerID="8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"
Sep 12 00:44:50.998129 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.998093    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0eb7ce6-1271-4e61-a562-44a0353d9def" containerID="62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725" exitCode=0
Sep 12 00:44:50.998200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.998151    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"
Sep 12 00:44:50.998200 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.998180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" event={"ID":"d0eb7ce6-1271-4e61-a562-44a0353d9def","Type":"ContainerDied","Data":"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"}
Sep 12 00:44:50.998307 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.998221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2" event={"ID":"d0eb7ce6-1271-4e61-a562-44a0353d9def","Type":"ContainerDied","Data":"b1cd5fea0fdc6d98dd4ca071a7fdb0e03d5e67fd9eb304f0b0bfe5749bfd2b57"}
Sep 12 00:44:50.999882 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.999859    2580 generic.go:356] "Generic (PLEG): container finished" podID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" containerID="068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5" exitCode=0
Sep 12 00:44:51.000006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.999926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"
Sep 12 00:44:51.000006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.999952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" event={"ID":"2b5ec12a-2718-47c1-bfb6-89e7ea157da1","Type":"ContainerDied","Data":"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"}
Sep 12 00:44:51.000006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:50.999985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7" event={"ID":"2b5ec12a-2718-47c1-bfb6-89e7ea157da1","Type":"ContainerDied","Data":"e8a0cc7123b33e949a43bfafeb9d3305e037039e799996d7f9c969d961a98bd7"}
Sep 12 00:44:51.008020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.007998    2580 scope.go:117] "RemoveContainer" containerID="8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"
Sep 12 00:44:51.008312 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:44:51.008284    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb\": container with ID starting with 8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb not found: ID does not exist" containerID="8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"
Sep 12 00:44:51.008407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.008320    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb"} err="failed to get container status \"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb\": rpc error: code = NotFound desc = could not find container \"8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb\": container with ID starting with 8bbb04011f5b15e6403ea8631a84badb53ec05590730326543387f041fbfc4bb not found: ID does not exist"
Sep 12 00:44:51.008407 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.008343    2580 scope.go:117] "RemoveContainer" containerID="62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"
Sep 12 00:44:51.016709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.016685    2580 scope.go:117] "RemoveContainer" containerID="62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"
Sep 12 00:44:51.017136 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:44:51.017080    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725\": container with ID starting with 62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725 not found: ID does not exist" containerID="62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"
Sep 12 00:44:51.017258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.017137    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725"} err="failed to get container status \"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725\": rpc error: code = NotFound desc = could not find container \"62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725\": container with ID starting with 62b2a035fc063711920cea7dc895ac90de74cea74fd9058724f3deebe8157725 not found: ID does not exist"
Sep 12 00:44:51.017258 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.017155    2580 scope.go:117] "RemoveContainer" containerID="068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"
Sep 12 00:44:51.020094 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.020070    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:44:51.024108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.024086    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-139288/premium-w4-139288-kserve-768c498bf7-czlnz"]
Sep 12 00:44:51.025705 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.025679    2580 scope.go:117] "RemoveContainer" containerID="068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"
Sep 12 00:44:51.025964 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:44:51.025946    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5\": container with ID starting with 068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5 not found: ID does not exist" containerID="068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"
Sep 12 00:44:51.026033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.025971    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5"} err="failed to get container status \"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5\": rpc error: code = NotFound desc = could not find container \"068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5\": container with ID starting with 068d2f2b9a0385ffa390f821ab65d5e0eb0a3d03e7d55d1d6accd841a50462a5 not found: ID does not exist"
Sep 12 00:44:51.037670 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.037648    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:44:51.043436 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.043385    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-w4-139288-kserve-6d4fbbfdd4-m6vq2"]
Sep 12 00:44:51.053491 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.053470    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:44:51.056437 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.056416    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-139288/distinct-2-w4-139288-kserve-57dd999bd7-256h7"]
Sep 12 00:44:51.933830 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:51.933803    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:44:52.374586 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:52.374506    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" path="/var/lib/kubelet/pods/2b5ec12a-2718-47c1-bfb6-89e7ea157da1/volumes"
Sep 12 00:44:52.374972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:52.374914    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" path="/var/lib/kubelet/pods/75b81ef8-fe97-48ef-b9c3-0e808f7d3809/volumes"
Sep 12 00:44:52.375273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:44:52.375259    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0eb7ce6-1271-4e61-a562-44a0353d9def" path="/var/lib/kubelet/pods/d0eb7ce6-1271-4e61-a562-44a0353d9def/volumes"
Sep 12 00:45:00.137002 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.136966    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"]
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137360    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137374    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137384    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137392    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137406    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0eb7ce6-1271-4e61-a562-44a0353d9def" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137411    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0eb7ce6-1271-4e61-a562-44a0353d9def" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137423    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddcc8f9-93b0-4222-af14-062f437c2b21" containerName="maas-api"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137428    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddcc8f9-93b0-4222-af14-062f437c2b21" containerName="maas-api"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137491    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0eb7ce6-1271-4e61-a562-44a0353d9def" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137502    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b81ef8-fe97-48ef-b9c3-0e808f7d3809" containerName="main"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137512    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ddcc8f9-93b0-4222-af14-062f437c2b21" containerName="maas-api"
Sep 12 00:45:00.137570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.137535    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b5ec12a-2718-47c1-bfb6-89e7ea157da1" containerName="main"
Sep 12 00:45:00.142142 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.142119    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"]
Sep 12 00:45:00.142249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.142219    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"
Sep 12 00:45:00.144337 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.144308    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:45:00.145777 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.145743    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"]
Sep 12 00:45:00.146116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.145954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"
Sep 12 00:45:00.150293 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.150275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"]
Sep 12 00:45:00.150820 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.150471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"
Sep 12 00:45:00.153314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.153145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"]
Sep 12 00:45:00.155946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.155924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"]
Sep 12 00:45:00.158556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.158536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"
Sep 12 00:45:00.160947 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.160926    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"]
Sep 12 00:45:00.165102 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.165083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"
Sep 12 00:45:00.169444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.169420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"
Sep 12 00:45:00.174108 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.173444    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"]
Sep 12 00:45:00.210076 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.210044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"
Sep 12 00:45:00.224573 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.224345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"
Sep 12 00:45:00.444731 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.444704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"]
Sep 12 00:45:00.445580 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:00.445554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91134443_0170_445e_8c6d_e53f1d25bcac.slice/crio-96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94 WatchSource:0}: Error finding container 96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94: Status 404 returned error can't find the container with id 96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94
Sep 12 00:45:00.469210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.469184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"]
Sep 12 00:45:00.480997 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:00.480971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c0ad9c_b5bc_4952_a690_2adb79dd2634.slice/crio-66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4 WatchSource:0}: Error finding container 66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4: Status 404 returned error can't find the container with id 66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4
Sep 12 00:45:00.515723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.515528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"]
Sep 12 00:45:00.518277 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:00.518252    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04542f81_532d_4071_9af2_884d58052712.slice/crio-5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342 WatchSource:0}: Error finding container 5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342: Status 404 returned error can't find the container with id 5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342
Sep 12 00:45:00.519476 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:00.519447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"]
Sep 12 00:45:00.521564 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:00.521542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b60b1ca_ec9e_49a8_88a0_5451e236aebf.slice/crio-dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e WatchSource:0}: Error finding container dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e: Status 404 returned error can't find the container with id dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e
Sep 12 00:45:01.060007 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.059905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7" event={"ID":"7b60b1ca-ec9e-49a8-88a0-5451e236aebf","Type":"ContainerStarted","Data":"2e0714ff632253d7c6703e6abf8797d6aec0b631c9b1cfcc9382aa5ad37affc4"}
Sep 12 00:45:01.060160 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.060010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7" event={"ID":"7b60b1ca-ec9e-49a8-88a0-5451e236aebf","Type":"ContainerStarted","Data":"dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e"}
Sep 12 00:45:01.061568 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.061538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj" event={"ID":"04542f81-532d-4071-9af2-884d58052712","Type":"ContainerStarted","Data":"16bce250146ef8def4ab793bf09f2d010540444f475abcd707e9b7bdfa6847b5"}
Sep 12 00:45:01.061715 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.061589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj" event={"ID":"04542f81-532d-4071-9af2-884d58052712","Type":"ContainerStarted","Data":"5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342"}
Sep 12 00:45:01.063079 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.063054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq" event={"ID":"91134443-0170-445e-8c6d-e53f1d25bcac","Type":"ContainerStarted","Data":"776ae5a2c835bfc69fa9e6df4bab831a799c67b2347a20c4c77e3269f80576b1"}
Sep 12 00:45:01.063203 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.063097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq" event={"ID":"91134443-0170-445e-8c6d-e53f1d25bcac","Type":"ContainerStarted","Data":"96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94"}
Sep 12 00:45:01.064561 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.064525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg" event={"ID":"90c0ad9c-b5bc-4952-a690-2adb79dd2634","Type":"ContainerStarted","Data":"210ca85e18231e79b9447d436ddfd5491ed0305e12ee2383db5357308270b7f2"}
Sep 12 00:45:01.064677 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.064580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg" event={"ID":"90c0ad9c-b5bc-4952-a690-2adb79dd2634","Type":"ContainerStarted","Data":"66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4"}
Sep 12 00:45:01.075681 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.075639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7" podStartSLOduration=1.07562624 podStartE2EDuration="1.07562624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.074267931 +0000 UTC m=+1697.208258155" watchObservedRunningTime="2026-09-12 00:45:01.07562624 +0000 UTC m=+1697.209616459"
Sep 12 00:45:01.087862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.087819    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq" podStartSLOduration=1.08780295 podStartE2EDuration="1.08780295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.086984324 +0000 UTC m=+1697.220974560" watchObservedRunningTime="2026-09-12 00:45:01.08780295 +0000 UTC m=+1697.221793176"
Sep 12 00:45:01.100075 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.100039    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg" podStartSLOduration=1.100026117 podStartE2EDuration="1.100026117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.0987039 +0000 UTC m=+1697.232694122" watchObservedRunningTime="2026-09-12 00:45:01.100026117 +0000 UTC m=+1697.234016388"
Sep 12 00:45:01.113340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:01.113300    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj" podStartSLOduration=1.1132886260000001 podStartE2EDuration="1.113288626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.110868464 +0000 UTC m=+1697.244858689" watchObservedRunningTime="2026-09-12 00:45:01.113288626 +0000 UTC m=+1697.247278850"
Sep 12 00:45:02.071701 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.071607    2580 generic.go:356] "Generic (PLEG): container finished" podID="7b60b1ca-ec9e-49a8-88a0-5451e236aebf" containerID="2e0714ff632253d7c6703e6abf8797d6aec0b631c9b1cfcc9382aa5ad37affc4" exitCode=0
Sep 12 00:45:02.072156 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.071703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7" event={"ID":"7b60b1ca-ec9e-49a8-88a0-5451e236aebf","Type":"ContainerDied","Data":"2e0714ff632253d7c6703e6abf8797d6aec0b631c9b1cfcc9382aa5ad37affc4"}
Sep 12 00:45:02.073248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.073224    2580 generic.go:356] "Generic (PLEG): container finished" podID="04542f81-532d-4071-9af2-884d58052712" containerID="16bce250146ef8def4ab793bf09f2d010540444f475abcd707e9b7bdfa6847b5" exitCode=0
Sep 12 00:45:02.073366 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.073298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj" event={"ID":"04542f81-532d-4071-9af2-884d58052712","Type":"ContainerDied","Data":"16bce250146ef8def4ab793bf09f2d010540444f475abcd707e9b7bdfa6847b5"}
Sep 12 00:45:02.074856 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.074836    2580 generic.go:356] "Generic (PLEG): container finished" podID="91134443-0170-445e-8c6d-e53f1d25bcac" containerID="776ae5a2c835bfc69fa9e6df4bab831a799c67b2347a20c4c77e3269f80576b1" exitCode=0
Sep 12 00:45:02.074940 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.074864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq" event={"ID":"91134443-0170-445e-8c6d-e53f1d25bcac","Type":"ContainerDied","Data":"776ae5a2c835bfc69fa9e6df4bab831a799c67b2347a20c4c77e3269f80576b1"}
Sep 12 00:45:02.076299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.076279    2580 generic.go:356] "Generic (PLEG): container finished" podID="90c0ad9c-b5bc-4952-a690-2adb79dd2634" containerID="210ca85e18231e79b9447d436ddfd5491ed0305e12ee2383db5357308270b7f2" exitCode=0
Sep 12 00:45:02.076375 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:02.076318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg" event={"ID":"90c0ad9c-b5bc-4952-a690-2adb79dd2634","Type":"ContainerDied","Data":"210ca85e18231e79b9447d436ddfd5491ed0305e12ee2383db5357308270b7f2"}
Sep 12 00:45:03.255804 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:03.255756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"
Sep 12 00:45:03.260473 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:03.260445    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"
Sep 12 00:45:03.286632 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:03.286604    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"
Sep 12 00:45:03.296809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:03.296746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"
Sep 12 00:45:04.091729 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.091701    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"
Sep 12 00:45:04.091948 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.091698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj" event={"ID":"04542f81-532d-4071-9af2-884d58052712","Type":"ContainerDied","Data":"5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342"}
Sep 12 00:45:04.091948 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.091837    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5df2972e2cf2af4d7d811636602cb42e3bb0dcd50c883d85e2a8d75989f69342"
Sep 12 00:45:04.093213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.093196    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"
Sep 12 00:45:04.093213 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.093207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq" event={"ID":"91134443-0170-445e-8c6d-e53f1d25bcac","Type":"ContainerDied","Data":"96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94"}
Sep 12 00:45:04.093352 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.093247    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96094e67ca64f4ae267718da84b03778eab97de8fea22cefe47fe458eb72dd94"
Sep 12 00:45:04.094619 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.094601    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"
Sep 12 00:45:04.094728 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.094650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg" event={"ID":"90c0ad9c-b5bc-4952-a690-2adb79dd2634","Type":"ContainerDied","Data":"66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4"}
Sep 12 00:45:04.094728 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.094691    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66e85af44a6e84c30e29d53b1db5c4611a15eb8e545f60303a2ec9dc165717e4"
Sep 12 00:45:04.096184 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.096162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7"
Sep 12 00:45:04.096277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.096165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819565-lchr7" event={"ID":"7b60b1ca-ec9e-49a8-88a0-5451e236aebf","Type":"ContainerDied","Data":"dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e"}
Sep 12 00:45:04.096277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:04.096271    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc6344b282d9f78e0b028a5082a8818924b2d06ca6b160ca0885c44f0ba4dd8e"
Sep 12 00:45:16.258555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.258462    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:45:16.259036 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.258723    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" podUID="d9710f59-139b-4720-8f2e-1c1fde01b94d" containerName="payload-processing" containerID="cri-o://9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269" gracePeriod=30
Sep 12 00:45:16.508861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.508797    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:45:16.607690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.607657    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume\") pod \"d9710f59-139b-4720-8f2e-1c1fde01b94d\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") "
Sep 12 00:45:16.607920 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.607702    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqt5w\" (UniqueName: \"kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w\") pod \"d9710f59-139b-4720-8f2e-1c1fde01b94d\" (UID: \"d9710f59-139b-4720-8f2e-1c1fde01b94d\") "
Sep 12 00:45:16.608133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.608108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9710f59-139b-4720-8f2e-1c1fde01b94d" (UID: "d9710f59-139b-4720-8f2e-1c1fde01b94d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:16.609828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.609800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w" (OuterVolumeSpecName: "kube-api-access-pqt5w") pod "d9710f59-139b-4720-8f2e-1c1fde01b94d" (UID: "d9710f59-139b-4720-8f2e-1c1fde01b94d"). InnerVolumeSpecName "kube-api-access-pqt5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:16.709080 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.709039    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9710f59-139b-4720-8f2e-1c1fde01b94d-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:16.709080 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:16.709071    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqt5w\" (UniqueName: \"kubernetes.io/projected/d9710f59-139b-4720-8f2e-1c1fde01b94d-kube-api-access-pqt5w\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:17.171709 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.171667    2580 generic.go:356] "Generic (PLEG): container finished" podID="d9710f59-139b-4720-8f2e-1c1fde01b94d" containerID="9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269" exitCode=0
Sep 12 00:45:17.171885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.171727    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"
Sep 12 00:45:17.171885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.171757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" event={"ID":"d9710f59-139b-4720-8f2e-1c1fde01b94d","Type":"ContainerDied","Data":"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"}
Sep 12 00:45:17.171885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.171851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg" event={"ID":"d9710f59-139b-4720-8f2e-1c1fde01b94d","Type":"ContainerDied","Data":"5c23d647dfabdf493d955004e8d8d21ab8cd651b560509e339fbbe23150521a5"}
Sep 12 00:45:17.171885 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.171874    2580 scope.go:117] "RemoveContainer" containerID="9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"
Sep 12 00:45:17.181459 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.181438    2580 scope.go:117] "RemoveContainer" containerID="9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"
Sep 12 00:45:17.181754 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:45:17.181734    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269\": container with ID starting with 9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269 not found: ID does not exist" containerID="9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"
Sep 12 00:45:17.181842 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.181779    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269"} err="failed to get container status \"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269\": rpc error: code = NotFound desc = could not find container \"9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269\": container with ID starting with 9ad9d540f0f5a08dea4840600ea0d075b2c74da7d83f5421066fa78bbc2c2269 not found: ID does not exist"
Sep 12 00:45:17.193458 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.193131    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:45:17.195787 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:17.195754    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-829959-68c9cb8cd7-7sbpg"]
Sep 12 00:45:18.375379 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:18.375345    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9710f59-139b-4720-8f2e-1c1fde01b94d" path="/var/lib/kubelet/pods/d9710f59-139b-4720-8f2e-1c1fde01b94d/volumes"
Sep 12 00:45:23.273727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.273686    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"]
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9710f59-139b-4720-8f2e-1c1fde01b94d" containerName="payload-processing"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274082    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9710f59-139b-4720-8f2e-1c1fde01b94d" containerName="payload-processing"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274091    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04542f81-532d-4071-9af2-884d58052712" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274097    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04542f81-532d-4071-9af2-884d58052712" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274115    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91134443-0170-445e-8c6d-e53f1d25bcac" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91134443-0170-445e-8c6d-e53f1d25bcac" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274126    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b60b1ca-ec9e-49a8-88a0-5451e236aebf" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274131    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b60b1ca-ec9e-49a8-88a0-5451e236aebf" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274137    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90c0ad9c-b5bc-4952-a690-2adb79dd2634" containerName="cleanup"
Sep 12 00:45:23.274162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274142    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90c0ad9c-b5bc-4952-a690-2adb79dd2634" containerName="cleanup"
Sep 12 00:45:23.274513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274203    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91134443-0170-445e-8c6d-e53f1d25bcac" containerName="cleanup"
Sep 12 00:45:23.274513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274209    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b60b1ca-ec9e-49a8-88a0-5451e236aebf" containerName="cleanup"
Sep 12 00:45:23.274513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274216    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9710f59-139b-4720-8f2e-1c1fde01b94d" containerName="payload-processing"
Sep 12 00:45:23.274513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274223    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04542f81-532d-4071-9af2-884d58052712" containerName="cleanup"
Sep 12 00:45:23.274513 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.274229    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90c0ad9c-b5bc-4952-a690-2adb79dd2634" containerName="cleanup"
Sep 12 00:45:23.278601 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.278582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:23.281870 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.281840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:45:23.284858 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.284832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"]
Sep 12 00:45:23.375875 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.375816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-593845-9glkz\" (UID: \"9b566af9-481b-4f19-9e82-f7e960f6c7b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:23.477006 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.476952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-593845-9glkz\" (UID: \"9b566af9-481b-4f19-9e82-f7e960f6c7b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:23.477637 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.477617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-593845-9glkz\" (UID: \"9b566af9-481b-4f19-9e82-f7e960f6c7b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:23.591651 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.591567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:23.863541 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:23.863510    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"]
Sep 12 00:45:23.866386 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:23.866357    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b566af9_481b_4f19_9e82_f7e960f6c7b6.slice/crio-35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d WatchSource:0}: Error finding container 35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d: Status 404 returned error can't find the container with id 35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d
Sep 12 00:45:24.215145 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:24.215105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz" event={"ID":"9b566af9-481b-4f19-9e82-f7e960f6c7b6","Type":"ContainerStarted","Data":"3af76f9bfcc0564c4b306758b64ab302d5cdaf3f1b157ee60fcb3ac4d21d5eb0"}
Sep 12 00:45:24.215330 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:24.215160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz" event={"ID":"9b566af9-481b-4f19-9e82-f7e960f6c7b6","Type":"ContainerStarted","Data":"35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d"}
Sep 12 00:45:24.228810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:24.228739    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz" podStartSLOduration=1.22872398 podStartE2EDuration="1.22872398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:24.22786339 +0000 UTC m=+1720.361853606" watchObservedRunningTime="2026-09-12 00:45:24.22872398 +0000 UTC m=+1720.362714206"
Sep 12 00:45:25.223085 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:25.223050    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b566af9-481b-4f19-9e82-f7e960f6c7b6" containerID="3af76f9bfcc0564c4b306758b64ab302d5cdaf3f1b157ee60fcb3ac4d21d5eb0" exitCode=0
Sep 12 00:45:25.223444 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:25.223119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz" event={"ID":"9b566af9-481b-4f19-9e82-f7e960f6c7b6","Type":"ContainerDied","Data":"3af76f9bfcc0564c4b306758b64ab302d5cdaf3f1b157ee60fcb3ac4d21d5eb0"}
Sep 12 00:45:26.367143 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:26.367115    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:26.505392 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:26.505302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca\") pod \"9b566af9-481b-4f19-9e82-f7e960f6c7b6\" (UID: \"9b566af9-481b-4f19-9e82-f7e960f6c7b6\") "
Sep 12 00:45:26.505710 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:26.505684    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b566af9-481b-4f19-9e82-f7e960f6c7b6" (UID: "9b566af9-481b-4f19-9e82-f7e960f6c7b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:26.606754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:26.606715    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b566af9-481b-4f19-9e82-f7e960f6c7b6-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:27.237114 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:27.237088    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"
Sep 12 00:45:27.237299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:27.237081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz" event={"ID":"9b566af9-481b-4f19-9e82-f7e960f6c7b6","Type":"ContainerDied","Data":"35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d"}
Sep 12 00:45:27.237299 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:27.237249    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35c36de0d445e9989bc70d21237603314ef78d1c1d6a28a4d0c449a21c4e4d8d"
Sep 12 00:45:38.925112 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:38.925073    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:45:38.925704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:38.925298    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" podUID="71e3f18a-92d9-45fb-804e-a1c2637ff793" containerName="payload-processing" containerID="cri-o://ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135" gracePeriod=30
Sep 12 00:45:39.073876 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.073837    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"]
Sep 12 00:45:39.077197 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.077168    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-593845-9glkz"]
Sep 12 00:45:39.169723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.169683    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:45:39.229865 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.229782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw8wd\" (UniqueName: \"kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd\") pod \"71e3f18a-92d9-45fb-804e-a1c2637ff793\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") "
Sep 12 00:45:39.230013 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.229918    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume\") pod \"71e3f18a-92d9-45fb-804e-a1c2637ff793\" (UID: \"71e3f18a-92d9-45fb-804e-a1c2637ff793\") "
Sep 12 00:45:39.230277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.230244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "71e3f18a-92d9-45fb-804e-a1c2637ff793" (UID: "71e3f18a-92d9-45fb-804e-a1c2637ff793"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:39.231788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.231740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd" (OuterVolumeSpecName: "kube-api-access-jw8wd") pod "71e3f18a-92d9-45fb-804e-a1c2637ff793" (UID: "71e3f18a-92d9-45fb-804e-a1c2637ff793"). InnerVolumeSpecName "kube-api-access-jw8wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:39.309165 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.309127    2580 generic.go:356] "Generic (PLEG): container finished" podID="71e3f18a-92d9-45fb-804e-a1c2637ff793" containerID="ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135" exitCode=0
Sep 12 00:45:39.309165 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.309166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" event={"ID":"71e3f18a-92d9-45fb-804e-a1c2637ff793","Type":"ContainerDied","Data":"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"}
Sep 12 00:45:39.309424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.309176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"
Sep 12 00:45:39.309424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.309208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg" event={"ID":"71e3f18a-92d9-45fb-804e-a1c2637ff793","Type":"ContainerDied","Data":"c545999c4c16b45ba4161a38f440998c541c12b9819f50f452ab401d56d76a99"}
Sep 12 00:45:39.309424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.309229    2580 scope.go:117] "RemoveContainer" containerID="ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"
Sep 12 00:45:39.319345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.319327    2580 scope.go:117] "RemoveContainer" containerID="ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"
Sep 12 00:45:39.319596 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:45:39.319577    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135\": container with ID starting with ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135 not found: ID does not exist" containerID="ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"
Sep 12 00:45:39.319646 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.319605    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135"} err="failed to get container status \"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135\": rpc error: code = NotFound desc = could not find container \"ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135\": container with ID starting with ada6e9ecc3a03da7bfe09d6aecd165414c195e59df8136f49688f9680d03f135 not found: ID does not exist"
Sep 12 00:45:39.336341 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.333073    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71e3f18a-92d9-45fb-804e-a1c2637ff793-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.336341 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.333116    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw8wd\" (UniqueName: \"kubernetes.io/projected/71e3f18a-92d9-45fb-804e-a1c2637ff793-kube-api-access-jw8wd\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.336341 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.336284    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:45:39.337702 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:39.337674    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-593845-6cbbb9c6b9-l96kg"]
Sep 12 00:45:40.374214 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:40.374180    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e3f18a-92d9-45fb-804e-a1c2637ff793" path="/var/lib/kubelet/pods/71e3f18a-92d9-45fb-804e-a1c2637ff793/volumes"
Sep 12 00:45:40.374595 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:40.374495    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b566af9-481b-4f19-9e82-f7e960f6c7b6" path="/var/lib/kubelet/pods/9b566af9-481b-4f19-9e82-f7e960f6c7b6/volumes"
Sep 12 00:45:44.612604 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:44.612573    2580 scope.go:117] "RemoveContainer" containerID="4d2a8bc55f7bf6b61e72337f1247259c2065ff13950262fc1663b4a30764344e"
Sep 12 00:45:45.849232 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849194    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"]
Sep 12 00:45:45.849625 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849604    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e3f18a-92d9-45fb-804e-a1c2637ff793" containerName="payload-processing"
Sep 12 00:45:45.849625 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849616    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e3f18a-92d9-45fb-804e-a1c2637ff793" containerName="payload-processing"
Sep 12 00:45:45.849699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849629    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b566af9-481b-4f19-9e82-f7e960f6c7b6" containerName="revoke-keys"
Sep 12 00:45:45.849699 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849636    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b566af9-481b-4f19-9e82-f7e960f6c7b6" containerName="revoke-keys"
Sep 12 00:45:45.849790 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849698    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b566af9-481b-4f19-9e82-f7e960f6c7b6" containerName="revoke-keys"
Sep 12 00:45:45.849790 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.849707    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e3f18a-92d9-45fb-804e-a1c2637ff793" containerName="payload-processing"
Sep 12 00:45:45.854806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.854787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:45.857090 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.857061    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:45:45.858926 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.858902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"]
Sep 12 00:45:45.894981 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.894950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt\" (UID: \"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:45.949847 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.949814    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7885-effda7"]
Sep 12 00:45:45.953277 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.953258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:45.955274 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.955250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qwc4m\""
Sep 12 00:45:45.960026 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.960003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-effda7"]
Sep 12 00:45:45.995815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.995755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt\" (UID: \"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:45.995990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.995862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f5kh\" (UniqueName: \"kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh\") pod \"test-curl-7885-effda7\" (UID: \"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5\") " pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:45.996370 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:45.996349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt\" (UID: \"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:46.097356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.097317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f5kh\" (UniqueName: \"kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh\") pod \"test-curl-7885-effda7\" (UID: \"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5\") " pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:46.105632 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.105572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f5kh\" (UniqueName: \"kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh\") pod \"test-curl-7885-effda7\" (UID: \"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5\") " pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:46.166816 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.166786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:46.264008 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.263974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:46.435698 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.435672    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"]
Sep 12 00:45:46.437265 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:46.437240    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbf8969_5d3e_4a50_95e4_c4ebd5da0673.slice/crio-3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f WatchSource:0}: Error finding container 3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f: Status 404 returned error can't find the container with id 3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f
Sep 12 00:45:46.532135 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:46.532110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-effda7"]
Sep 12 00:45:46.533551 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:45:46.533520    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55acf06d_06aa_4dbc_87f1_36ac2c3d54b5.slice/crio-2733830feea0949ec02336d780c746f1e507d4204dff63e04b9ebf1df1a57cce WatchSource:0}: Error finding container 2733830feea0949ec02336d780c746f1e507d4204dff63e04b9ebf1df1a57cce: Status 404 returned error can't find the container with id 2733830feea0949ec02336d780c746f1e507d4204dff63e04b9ebf1df1a57cce
Sep 12 00:45:47.359469 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.359380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-effda7" event={"ID":"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5","Type":"ContainerStarted","Data":"eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f"}
Sep 12 00:45:47.359469 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.359418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-effda7" event={"ID":"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5","Type":"ContainerStarted","Data":"2733830feea0949ec02336d780c746f1e507d4204dff63e04b9ebf1df1a57cce"}
Sep 12 00:45:47.360981 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.360954    2580 generic.go:356] "Generic (PLEG): container finished" podID="bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" containerID="aed8c9212b2f01f9ace25e954ab002b3879f8974b98a6cf0597a716adf11463c" exitCode=0
Sep 12 00:45:47.361101 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.361084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt" event={"ID":"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673","Type":"ContainerDied","Data":"aed8c9212b2f01f9ace25e954ab002b3879f8974b98a6cf0597a716adf11463c"}
Sep 12 00:45:47.361153 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.361125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt" event={"ID":"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673","Type":"ContainerStarted","Data":"3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f"}
Sep 12 00:45:47.371527 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:47.371464    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7885-effda7" podStartSLOduration=2.371446015 podStartE2EDuration="2.371446015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:47.371138481 +0000 UTC m=+1743.505128708" watchObservedRunningTime="2026-09-12 00:45:47.371446015 +0000 UTC m=+1743.505436240"
Sep 12 00:45:48.018565 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.018521    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7885-effda7"]
Sep 12 00:45:48.022598 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.022569    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7885-effda7"]
Sep 12 00:45:48.367745 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:45:48.367581    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7885-effda7" container="test-curl-7885-effda7"
Sep 12 00:45:48.525305 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.525282    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:48.621521 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.621415    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca\") pod \"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673\" (UID: \"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673\") "
Sep 12 00:45:48.621808 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.621758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" (UID: "bcbf8969-5d3e-4a50-95e4-c4ebd5da0673"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:48.722688 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:48.722647    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:49.375024 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:49.374998    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"
Sep 12 00:45:49.375424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:49.375043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt" event={"ID":"bcbf8969-5d3e-4a50-95e4-c4ebd5da0673","Type":"ContainerDied","Data":"3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f"}
Sep 12 00:45:49.375424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:49.375088    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dd20a659f9aefee3c334e07173c43f8f0e6132a8922b72f39c112ce3a43732f"
Sep 12 00:45:49.375424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:49.375240    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7885-effda7" podUID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" containerName="test-curl-7885-effda7" containerID="cri-o://eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f" gracePeriod=2
Sep 12 00:45:51.541074 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:51.541049    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:51.651361 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:51.651278    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f5kh\" (UniqueName: \"kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh\") pod \"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5\" (UID: \"55acf06d-06aa-4dbc-87f1-36ac2c3d54b5\") "
Sep 12 00:45:51.653351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:51.653320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh" (OuterVolumeSpecName: "kube-api-access-9f5kh") pod "55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" (UID: "55acf06d-06aa-4dbc-87f1-36ac2c3d54b5"). InnerVolumeSpecName "kube-api-access-9f5kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:51.752921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:51.752885    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f5kh\" (UniqueName: \"kubernetes.io/projected/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5-kube-api-access-9f5kh\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:52.373611 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.373576    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" path="/var/lib/kubelet/pods/55acf06d-06aa-4dbc-87f1-36ac2c3d54b5/volumes"
Sep 12 00:45:52.393431 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.393401    2580 generic.go:356] "Generic (PLEG): container finished" podID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" containerID="eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f" exitCode=137
Sep 12 00:45:52.393606 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.393441    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-effda7"
Sep 12 00:45:52.393606 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.393503    2580 scope.go:117] "RemoveContainer" containerID="eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f"
Sep 12 00:45:52.402563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.402543    2580 scope.go:117] "RemoveContainer" containerID="eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f"
Sep 12 00:45:52.402859 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:45:52.402839    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f\": container with ID starting with eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f not found: ID does not exist" containerID="eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f"
Sep 12 00:45:52.402936 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:52.402873    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f"} err="failed to get container status \"eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f\": rpc error: code = NotFound desc = could not find container \"eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f\": container with ID starting with eb1456f1cc7899b1deb5a475242ad74bbde2b06b0a93ea7fb5cb40edfc0aa75f not found: ID does not exist"
Sep 12 00:45:56.300381 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.300350    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"]
Sep 12 00:45:56.307106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.307080    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0d3a66-29819565-w2tdq"]
Sep 12 00:45:56.334498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.334465    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:45:56.334810 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.334759    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" podUID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" containerName="payload-processing" containerID="cri-o://2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068" gracePeriod=30
Sep 12 00:45:56.374458 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.374416    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91134443-0170-445e-8c6d-e53f1d25bcac" path="/var/lib/kubelet/pods/91134443-0170-445e-8c6d-e53f1d25bcac/volumes"
Sep 12 00:45:56.484174 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.484136    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"]
Sep 12 00:45:56.486941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.486915    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0d3a66-kmrtt"]
Sep 12 00:45:56.589642 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.589616    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:45:56.695509 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.695464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume\") pod \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") "
Sep 12 00:45:56.695683 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.695530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgtxk\" (UniqueName: \"kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk\") pod \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\" (UID: \"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1\") "
Sep 12 00:45:56.695903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.695880    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" (UID: "0bfa2ba7-223c-4b8d-a823-459c6ffa37d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:56.697660 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.697642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk" (OuterVolumeSpecName: "kube-api-access-tgtxk") pod "0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" (UID: "0bfa2ba7-223c-4b8d-a823-459c6ffa37d1"). InnerVolumeSpecName "kube-api-access-tgtxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:56.796964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.796918    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:56.796964 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:56.796955    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgtxk\" (UniqueName: \"kubernetes.io/projected/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1-kube-api-access-tgtxk\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:45:57.431803 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.431748    2580 generic.go:356] "Generic (PLEG): container finished" podID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" containerID="2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068" exitCode=0
Sep 12 00:45:57.432249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.431814    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"
Sep 12 00:45:57.432249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.431821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" event={"ID":"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1","Type":"ContainerDied","Data":"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"}
Sep 12 00:45:57.432249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.431864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq" event={"ID":"0bfa2ba7-223c-4b8d-a823-459c6ffa37d1","Type":"ContainerDied","Data":"5ee8d43d0ff90c7f46d36558190c1e97f442887c233c7ef70fbc73508e87a2ad"}
Sep 12 00:45:57.432249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.431886    2580 scope.go:117] "RemoveContainer" containerID="2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"
Sep 12 00:45:57.441088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.441068    2580 scope.go:117] "RemoveContainer" containerID="2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"
Sep 12 00:45:57.441348 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:45:57.441330    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068\": container with ID starting with 2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068 not found: ID does not exist" containerID="2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"
Sep 12 00:45:57.441400 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.441355    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068"} err="failed to get container status \"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068\": rpc error: code = NotFound desc = could not find container \"2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068\": container with ID starting with 2d120f6a74d89ed4412d5304a7644d825b1abebb3a1978cd989b93b3a4820068 not found: ID does not exist"
Sep 12 00:45:57.452602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.452577    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:45:57.459462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:57.459437    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0d3a66-854748765c-4g6kq"]
Sep 12 00:45:58.374621 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:58.374585    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" path="/var/lib/kubelet/pods/0bfa2ba7-223c-4b8d-a823-459c6ffa37d1/volumes"
Sep 12 00:45:58.374961 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:45:58.374948    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" path="/var/lib/kubelet/pods/bcbf8969-5d3e-4a50-95e4-c4ebd5da0673/volumes"
Sep 12 00:46:44.664788 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:46:44.664729    2580 scope.go:117] "RemoveContainer" containerID="776a3495fefa19f1e08618b8f2a8e2061c9f28225cd8c81124f7148e874eb4d8"
Sep 12 00:47:42.341836 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.341796    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7885-b6c83a"]
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342150    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" containerName="test-curl-7885-effda7"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342162    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" containerName="test-curl-7885-effda7"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342173    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" containerName="revoke-keys"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342179    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" containerName="revoke-keys"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342187    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" containerName="payload-processing"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342192    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" containerName="payload-processing"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342258    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55acf06d-06aa-4dbc-87f1-36ac2c3d54b5" containerName="test-curl-7885-effda7"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342268    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bfa2ba7-223c-4b8d-a823-459c6ffa37d1" containerName="payload-processing"
Sep 12 00:47:42.342340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.342276    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbf8969-5d3e-4a50-95e4-c4ebd5da0673" containerName="revoke-keys"
Sep 12 00:47:42.345215 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.345197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:42.347708 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.347685    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qwc4m\""
Sep 12 00:47:42.355515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.355494    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-b6c83a"]
Sep 12 00:47:42.479250 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.479199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw9qs\" (UniqueName: \"kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs\") pod \"test-curl-7885-b6c83a\" (UID: \"90ff9c46-8da1-46ac-b66a-a15817f07795\") " pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:42.580492 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.580451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw9qs\" (UniqueName: \"kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs\") pod \"test-curl-7885-b6c83a\" (UID: \"90ff9c46-8da1-46ac-b66a-a15817f07795\") " pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:42.588144 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.588119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw9qs\" (UniqueName: \"kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs\") pod \"test-curl-7885-b6c83a\" (UID: \"90ff9c46-8da1-46ac-b66a-a15817f07795\") " pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:42.655734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.655632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:42.927878 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:42.927852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-b6c83a"]
Sep 12 00:47:42.929869 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:47:42.929839    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90ff9c46_8da1_46ac_b66a_a15817f07795.slice/crio-1390beb9b9cd842f7a9ae3de793d80cf4ff3fe47c8f03f03ad9adea5cadedccc WatchSource:0}: Error finding container 1390beb9b9cd842f7a9ae3de793d80cf4ff3fe47c8f03f03ad9adea5cadedccc: Status 404 returned error can't find the container with id 1390beb9b9cd842f7a9ae3de793d80cf4ff3fe47c8f03f03ad9adea5cadedccc
Sep 12 00:47:43.026356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.026322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-b6c83a" event={"ID":"90ff9c46-8da1-46ac-b66a-a15817f07795","Type":"ContainerStarted","Data":"74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d"}
Sep 12 00:47:43.026356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.026368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-b6c83a" event={"ID":"90ff9c46-8da1-46ac-b66a-a15817f07795","Type":"ContainerStarted","Data":"1390beb9b9cd842f7a9ae3de793d80cf4ff3fe47c8f03f03ad9adea5cadedccc"}
Sep 12 00:47:43.042198 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.042145    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7885-b6c83a" podStartSLOduration=1.042129113 podStartE2EDuration="1.042129113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:47:43.039889213 +0000 UTC m=+1859.173879439" watchObservedRunningTime="2026-09-12 00:47:43.042129113 +0000 UTC m=+1859.176119338"
Sep 12 00:47:43.638010 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.637922    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7885-b6c83a"]
Sep 12 00:47:43.641897 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.641871    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7885-b6c83a"]
Sep 12 00:47:43.764012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.763975    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7885-023511"]
Sep 12 00:47:43.764384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.764371    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" containerName="test-curl-7885-b6c83a"
Sep 12 00:47:43.764430 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.764386    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" containerName="test-curl-7885-b6c83a"
Sep 12 00:47:43.764475 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.764466    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" containerName="test-curl-7885-b6c83a"
Sep 12 00:47:43.767592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.767574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:43.773784 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.773744    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-023511"]
Sep 12 00:47:43.894152 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.894054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwd6x\" (UniqueName: \"kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x\") pod \"test-curl-7885-023511\" (UID: \"31166031-8dff-439c-83d1-57f148e40482\") " pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:43.994670 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:43.994632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwd6x\" (UniqueName: \"kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x\") pod \"test-curl-7885-023511\" (UID: \"31166031-8dff-439c-83d1-57f148e40482\") " pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:44.003516 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:44.003481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwd6x\" (UniqueName: \"kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x\") pod \"test-curl-7885-023511\" (UID: \"31166031-8dff-439c-83d1-57f148e40482\") " pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:44.033199 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:44.033168    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7885-b6c83a" container="test-curl-7885-b6c83a"
Sep 12 00:47:44.077990 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:44.077940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:44.348017 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:44.347990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-023511"]
Sep 12 00:47:44.349821 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:47:44.349793    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31166031_8dff_439c_83d1_57f148e40482.slice/crio-26c89fd4e7a2368ea81e55ba962af0473d237ece5164c0b540efc15f200693d1 WatchSource:0}: Error finding container 26c89fd4e7a2368ea81e55ba962af0473d237ece5164c0b540efc15f200693d1: Status 404 returned error can't find the container with id 26c89fd4e7a2368ea81e55ba962af0473d237ece5164c0b540efc15f200693d1
Sep 12 00:47:44.695860 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:44.695829    2580 scope.go:117] "RemoveContainer" containerID="2979b7e6879883b75199748e4e24d5f0616799e35b26dcb87ede1dd15d453b18"
Sep 12 00:47:44.704676 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:44.704652    2580 scope.go:117] "RemoveContainer" containerID="a3f23834cd3e6f3f25bc55e0c7f0340c0ac65e4c3f131825d232c89873f5e201"
Sep 12 00:47:45.039794 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.039665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-023511" event={"ID":"31166031-8dff-439c-83d1-57f148e40482","Type":"ContainerStarted","Data":"7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c"}
Sep 12 00:47:45.039794 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.039720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-023511" event={"ID":"31166031-8dff-439c-83d1-57f148e40482","Type":"ContainerStarted","Data":"26c89fd4e7a2368ea81e55ba962af0473d237ece5164c0b540efc15f200693d1"}
Sep 12 00:47:45.040012 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.039809    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7885-b6c83a" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" containerName="test-curl-7885-b6c83a" containerID="cri-o://74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d" gracePeriod=2
Sep 12 00:47:45.055246 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.055194    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7885-023511" podStartSLOduration=2.055179352 podStartE2EDuration="2.055179352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:47:45.053425566 +0000 UTC m=+1861.187415794" watchObservedRunningTime="2026-09-12 00:47:45.055179352 +0000 UTC m=+1861.189169576"
Sep 12 00:47:45.666360 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.666321    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7885-023511"]
Sep 12 00:47:45.668821 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:45.668786    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7885-023511"]
Sep 12 00:47:46.045110 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:46.045073    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7885-023511" container="test-curl-7885-023511"
Sep 12 00:47:47.050350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:47.050315    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7885-023511" podUID="31166031-8dff-439c-83d1-57f148e40482" containerName="test-curl-7885-023511" containerID="cri-o://7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c" gracePeriod=2
Sep 12 00:47:47.205384 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:47.205359    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:47.222248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:47.222220    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw9qs\" (UniqueName: \"kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs\") pod \"90ff9c46-8da1-46ac-b66a-a15817f07795\" (UID: \"90ff9c46-8da1-46ac-b66a-a15817f07795\") "
Sep 12 00:47:47.224243 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:47.224207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs" (OuterVolumeSpecName: "kube-api-access-bw9qs") pod "90ff9c46-8da1-46ac-b66a-a15817f07795" (UID: "90ff9c46-8da1-46ac-b66a-a15817f07795"). InnerVolumeSpecName "kube-api-access-bw9qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:47:47.323424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:47.323342    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw9qs\" (UniqueName: \"kubernetes.io/projected/90ff9c46-8da1-46ac-b66a-a15817f07795-kube-api-access-bw9qs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:47:48.056924 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.056889    2580 generic.go:356] "Generic (PLEG): container finished" podID="90ff9c46-8da1-46ac-b66a-a15817f07795" containerID="74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d" exitCode=137
Sep 12 00:47:48.057359 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.056933    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:48.057359 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.056956    2580 scope.go:117] "RemoveContainer" containerID="74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d"
Sep 12 00:47:48.066822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.066801    2580 scope.go:117] "RemoveContainer" containerID="74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d"
Sep 12 00:47:48.067084 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:48.067066    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d\": container with ID starting with 74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d not found: ID does not exist" containerID="74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d"
Sep 12 00:47:48.067158 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.067094    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d"} err="failed to get container status \"74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d\": rpc error: code = NotFound desc = could not find container \"74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d\": container with ID starting with 74ad0f18e4bd5d37c4fb89951f4d03d4bb14cbb9eba9f45e4b18dc012f30729d not found: ID does not exist"
Sep 12 00:47:48.067666 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:48.067643    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-b6c83a\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" pod="opendatahub/test-curl-7885-b6c83a"
Sep 12 00:47:48.374326 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:48.374247    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90ff9c46-8da1-46ac-b66a-a15817f07795" path="/var/lib/kubelet/pods/90ff9c46-8da1-46ac-b66a-a15817f07795/volumes"
Sep 12 00:47:49.241716 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:49.241693    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:49.342741 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:49.342651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwd6x\" (UniqueName: \"kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x\") pod \"31166031-8dff-439c-83d1-57f148e40482\" (UID: \"31166031-8dff-439c-83d1-57f148e40482\") "
Sep 12 00:47:49.344722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:49.344698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x" (OuterVolumeSpecName: "kube-api-access-mwd6x") pod "31166031-8dff-439c-83d1-57f148e40482" (UID: "31166031-8dff-439c-83d1-57f148e40482"). InnerVolumeSpecName "kube-api-access-mwd6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:47:49.443378 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:49.443341    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwd6x\" (UniqueName: \"kubernetes.io/projected/31166031-8dff-439c-83d1-57f148e40482-kube-api-access-mwd6x\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:47:50.076325 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.076290    2580 generic.go:356] "Generic (PLEG): container finished" podID="31166031-8dff-439c-83d1-57f148e40482" containerID="7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c" exitCode=137
Sep 12 00:47:50.076531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.076339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:50.076531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.076393    2580 scope.go:117] "RemoveContainer" containerID="7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c"
Sep 12 00:47:50.085364 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.085347    2580 scope.go:117] "RemoveContainer" containerID="7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c"
Sep 12 00:47:50.085680 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:50.085651    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c\": container with ID starting with 7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c not found: ID does not exist" containerID="7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c"
Sep 12 00:47:50.085754 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.085683    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c"} err="failed to get container status \"7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c\": rpc error: code = NotFound desc = could not find container \"7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c\": container with ID starting with 7ab9d39e36e2b7ce54f281578d60008c13a1c7689e97a8e36eec8ff2b0e1a41c not found: ID does not exist"
Sep 12 00:47:50.086226 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:47:50.086203    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-023511\" is forbidden: User \"system:node:ip-10-0-134-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-179.ec2.internal' and this object" podUID="31166031-8dff-439c-83d1-57f148e40482" pod="opendatahub/test-curl-7885-023511"
Sep 12 00:47:50.374147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:47:50.374069    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31166031-8dff-439c-83d1-57f148e40482" path="/var/lib/kubelet/pods/31166031-8dff-439c-83d1-57f148e40482/volumes"
Sep 12 00:48:58.242212 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.242177    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:48:58.242641 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.242575    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31166031-8dff-439c-83d1-57f148e40482" containerName="test-curl-7885-023511"
Sep 12 00:48:58.242641 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.242588    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31166031-8dff-439c-83d1-57f148e40482" containerName="test-curl-7885-023511"
Sep 12 00:48:58.242715 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.242670    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31166031-8dff-439c-83d1-57f148e40482" containerName="test-curl-7885-023511"
Sep 12 00:48:58.245911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.245893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.248237 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.248213    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-64e462\""
Sep 12 00:48:58.248359 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.248285    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-64e462-dockercfg-fvmxn\""
Sep 12 00:48:58.257410 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.257384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:48:58.400817 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.400756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.401018 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.400889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvq6k\" (UniqueName: \"kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.501574 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.501476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvq6k\" (UniqueName: \"kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.501809 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.501623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.502401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.502381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.511592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.511568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvq6k\" (UniqueName: \"kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k\") pod \"payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.556555 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.556511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:58.826610 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:58.826527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:48:58.829686 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:48:58.829653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba33141_3031_44b7_b8d4_1c62cfc8b26d.slice/crio-c5c36a3849b3ffd0c3579a707008ef39d233f8a85e690604e1642160d379a54d WatchSource:0}: Error finding container c5c36a3849b3ffd0c3579a707008ef39d233f8a85e690604e1642160d379a54d: Status 404 returned error can't find the container with id c5c36a3849b3ffd0c3579a707008ef39d233f8a85e690604e1642160d379a54d
Sep 12 00:48:59.462345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:59.462312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" event={"ID":"eba33141-3031-44b7-b8d4-1c62cfc8b26d","Type":"ContainerStarted","Data":"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"}
Sep 12 00:48:59.462345 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:59.462362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" event={"ID":"eba33141-3031-44b7-b8d4-1c62cfc8b26d","Type":"ContainerStarted","Data":"c5c36a3849b3ffd0c3579a707008ef39d233f8a85e690604e1642160d379a54d"}
Sep 12 00:48:59.462904 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:59.462412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:48:59.480567 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:48:59.480505    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" podStartSLOduration=1.480489156 podStartE2EDuration="1.480489156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:48:59.477208333 +0000 UTC m=+1935.611198559" watchObservedRunningTime="2026-09-12 00:48:59.480489156 +0000 UTC m=+1935.614479382"
Sep 12 00:49:10.470592 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:49:10.470558    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:50:17.027956 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.027916    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:50:17.031626 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.031602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.035056 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.035028    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-64e462\"/\"kube-root-ca.crt\""
Sep 12 00:50:17.035230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.035082    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-64e462\"/\"openshift-service-ca.crt\""
Sep 12 00:50:17.035230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.035120    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-64e462\"/\"ipp-route-64e462-kserve-self-signed-certs\""
Sep 12 00:50:17.035230 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.035188    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-64e462\"/\"default-dockercfg-jwtrm\""
Sep 12 00:50:17.040419 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.040396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:50:17.132609 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.132822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.132822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.132822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.132822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.132974 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.132840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svmrj\" (UniqueName: \"kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svmrj\" (UniqueName: \"kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233723 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233797 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233901 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.233966 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.233919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.235698 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.235669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.235853 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.235822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.241778 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.241734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svmrj\" (UniqueName: \"kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj\") pod \"ipp-route-64e462-kserve-6c7bc69bff-sflnj\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") " pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.342515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.342411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:17.616233 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.616206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:50:17.618143 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:50:17.618111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf494e1e9_f464_4b91_a807_38ada37f4adf.slice/crio-529cb5da32e73415208d509a32c6efdcf95792694d163187daf0509f9de73f49 WatchSource:0}: Error finding container 529cb5da32e73415208d509a32c6efdcf95792694d163187daf0509f9de73f49: Status 404 returned error can't find the container with id 529cb5da32e73415208d509a32c6efdcf95792694d163187daf0509f9de73f49
Sep 12 00:50:17.928127 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.928092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" event={"ID":"f494e1e9-f464-4b91-a807-38ada37f4adf","Type":"ContainerStarted","Data":"57154ec61e3dce2aa980cd148bf71e191f030c0fc903f8c8bd8b11b62305db5f"}
Sep 12 00:50:17.928316 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.928165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" event={"ID":"f494e1e9-f464-4b91-a807-38ada37f4adf","Type":"ContainerStarted","Data":"529cb5da32e73415208d509a32c6efdcf95792694d163187daf0509f9de73f49"}
Sep 12 00:50:17.945262 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:17.945206    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" podStartSLOduration=0.945190135 podStartE2EDuration="945.190135ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:50:17.943461447 +0000 UTC m=+2014.077451671" watchObservedRunningTime="2026-09-12 00:50:17.945190135 +0000 UTC m=+2014.079180398"
Sep 12 00:50:27.343566 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:27.343525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:27.344095 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:27.343805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:27.356678 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:27.356652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:28.005500 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:28.005463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:50:46.580617 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:50:46.580582    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h_eba33141-3031-44b7-b8d4-1c62cfc8b26d/payload-processing/0.log"
Sep 12 00:51:16.318480 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.318396    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"]
Sep 12 00:51:16.322106 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.322083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:16.324351 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.324326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:51:16.329327 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.329286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"]
Sep 12 00:51:16.490465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.490426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h\" (UID: \"b793759c-83ae-4f0f-8e4c-99843832fddd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:16.591941 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.591840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h\" (UID: \"b793759c-83ae-4f0f-8e4c-99843832fddd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:16.592519 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.592498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h\" (UID: \"b793759c-83ae-4f0f-8e4c-99843832fddd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:16.634931 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.634878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:16.907911 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:16.907883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"]
Sep 12 00:51:16.909391 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:51:16.909358    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb793759c_83ae_4f0f_8e4c_99843832fddd.slice/crio-b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245 WatchSource:0}: Error finding container b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245: Status 404 returned error can't find the container with id b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245
Sep 12 00:51:17.290498 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:17.290462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h" event={"ID":"b793759c-83ae-4f0f-8e4c-99843832fddd","Type":"ContainerStarted","Data":"8470a879a773a521d607af5aa24c0f6fc52257be1e051c88c91a89b34e9040ac"}
Sep 12 00:51:17.290686 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:17.290545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h" event={"ID":"b793759c-83ae-4f0f-8e4c-99843832fddd","Type":"ContainerStarted","Data":"b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245"}
Sep 12 00:51:17.304901 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:17.304853    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h" podStartSLOduration=1.3048378440000001 podStartE2EDuration="1.304837844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:51:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:51:17.30298644 +0000 UTC m=+2073.436976666" watchObservedRunningTime="2026-09-12 00:51:17.304837844 +0000 UTC m=+2073.438828069"
Sep 12 00:51:18.297937 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:18.297901    2580 generic.go:356] "Generic (PLEG): container finished" podID="b793759c-83ae-4f0f-8e4c-99843832fddd" containerID="8470a879a773a521d607af5aa24c0f6fc52257be1e051c88c91a89b34e9040ac" exitCode=0
Sep 12 00:51:18.298340 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:18.297949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h" event={"ID":"b793759c-83ae-4f0f-8e4c-99843832fddd","Type":"ContainerDied","Data":"8470a879a773a521d607af5aa24c0f6fc52257be1e051c88c91a89b34e9040ac"}
Sep 12 00:51:19.430240 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:19.430211    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:19.619032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:19.618936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca\") pod \"b793759c-83ae-4f0f-8e4c-99843832fddd\" (UID: \"b793759c-83ae-4f0f-8e4c-99843832fddd\") "
Sep 12 00:51:19.619278 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:19.619252    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b793759c-83ae-4f0f-8e4c-99843832fddd" (UID: "b793759c-83ae-4f0f-8e4c-99843832fddd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:19.720761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:19.720723    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b793759c-83ae-4f0f-8e4c-99843832fddd-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:51:20.310910 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:20.310872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h" event={"ID":"b793759c-83ae-4f0f-8e4c-99843832fddd","Type":"ContainerDied","Data":"b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245"}
Sep 12 00:51:20.311135 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:20.310925    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b307866a75f53e087354dddc328c61218c9002bab37b3e2d006178ed1eca5245"
Sep 12 00:51:20.311135 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:20.310886    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"
Sep 12 00:51:22.691219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:22.691168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"]
Sep 12 00:51:22.694074 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:22.694028    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b4b570-np58h"]
Sep 12 00:51:24.374829 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:24.374794    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b793759c-83ae-4f0f-8e4c-99843832fddd" path="/var/lib/kubelet/pods/b793759c-83ae-4f0f-8e4c-99843832fddd/volumes"
Sep 12 00:51:33.034752 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.034714    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"]
Sep 12 00:51:33.035161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.035130    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b793759c-83ae-4f0f-8e4c-99843832fddd" containerName="revoke-keys"
Sep 12 00:51:33.035161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.035143    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b793759c-83ae-4f0f-8e4c-99843832fddd" containerName="revoke-keys"
Sep 12 00:51:33.035234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.035227    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b793759c-83ae-4f0f-8e4c-99843832fddd" containerName="revoke-keys"
Sep 12 00:51:33.038399 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.038381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:33.040689 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.040667    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:51:33.045133 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.045096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"]
Sep 12 00:51:33.146045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.146012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w\" (UID: \"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:33.247470 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.247438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w\" (UID: \"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:33.248137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.248112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w\" (UID: \"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:33.351556 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.351472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:33.621872 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:33.621845    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"]
Sep 12 00:51:33.623542 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:51:33.623513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4f6727_37e5_4f6c_8c92_5ec042bdc0b9.slice/crio-05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692 WatchSource:0}: Error finding container 05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692: Status 404 returned error can't find the container with id 05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692
Sep 12 00:51:34.393261 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:34.393228    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" containerID="b60cc0125b47418f29c1d4b39bd0d36774f50e5d22b4d53fe127a14a0bc7efc0" exitCode=0
Sep 12 00:51:34.393648 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:34.393313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w" event={"ID":"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9","Type":"ContainerDied","Data":"b60cc0125b47418f29c1d4b39bd0d36774f50e5d22b4d53fe127a14a0bc7efc0"}
Sep 12 00:51:34.393648 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:34.393365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w" event={"ID":"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9","Type":"ContainerStarted","Data":"05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692"}
Sep 12 00:51:35.550570 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:35.550544    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:35.670820 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:35.670790    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca\") pod \"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9\" (UID: \"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9\") "
Sep 12 00:51:35.671139 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:35.671109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" (UID: "9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:35.771751 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:35.771717    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9-maas-api-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:51:36.407629 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:36.407587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w" event={"ID":"9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9","Type":"ContainerDied","Data":"05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692"}
Sep 12 00:51:36.407831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:36.407660    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05b8f3fbca4c31dc690ec612dd2ac96f170b2e8f8cb7af3b66df8057db553692"
Sep 12 00:51:36.407831 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:36.407599    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"
Sep 12 00:51:44.763344 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:44.763310    2580 scope.go:117] "RemoveContainer" containerID="3af76f9bfcc0564c4b306758b64ab302d5cdaf3f1b157ee60fcb3ac4d21d5eb0"
Sep 12 00:51:44.772147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:44.772123    2580 scope.go:117] "RemoveContainer" containerID="776ae5a2c835bfc69fa9e6df4bab831a799c67b2347a20c4c77e3269f80576b1"
Sep 12 00:51:48.861985 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:48.861950    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:51:48.862486 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:48.862200    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" podUID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" containerName="payload-processing" containerID="cri-o://80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4" gracePeriod=30
Sep 12 00:51:49.065135 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.065099    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"]
Sep 12 00:51:49.070668 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.070626    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64e462-ll49w"]
Sep 12 00:51:49.121922 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.121862    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:51:49.198762 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.198720    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvq6k\" (UniqueName: \"kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k\") pod \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") "
Sep 12 00:51:49.198938 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.198850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume\") pod \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\" (UID: \"eba33141-3031-44b7-b8d4-1c62cfc8b26d\") "
Sep 12 00:51:49.199263 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.199233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eba33141-3031-44b7-b8d4-1c62cfc8b26d" (UID: "eba33141-3031-44b7-b8d4-1c62cfc8b26d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:49.200835 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.200794    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k" (OuterVolumeSpecName: "kube-api-access-kvq6k") pod "eba33141-3031-44b7-b8d4-1c62cfc8b26d" (UID: "eba33141-3031-44b7-b8d4-1c62cfc8b26d"). InnerVolumeSpecName "kube-api-access-kvq6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:49.300386 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.300354    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvq6k\" (UniqueName: \"kubernetes.io/projected/eba33141-3031-44b7-b8d4-1c62cfc8b26d-kube-api-access-kvq6k\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:51:49.300386 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.300380    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eba33141-3031-44b7-b8d4-1c62cfc8b26d-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:51:49.494697 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.494656    2580 generic.go:356] "Generic (PLEG): container finished" podID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" containerID="80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4" exitCode=0
Sep 12 00:51:49.494935 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.494712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"
Sep 12 00:51:49.494935 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.494737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" event={"ID":"eba33141-3031-44b7-b8d4-1c62cfc8b26d","Type":"ContainerDied","Data":"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"}
Sep 12 00:51:49.494935 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.494845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h" event={"ID":"eba33141-3031-44b7-b8d4-1c62cfc8b26d","Type":"ContainerDied","Data":"c5c36a3849b3ffd0c3579a707008ef39d233f8a85e690604e1642160d379a54d"}
Sep 12 00:51:49.494935 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.494879    2580 scope.go:117] "RemoveContainer" containerID="80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"
Sep 12 00:51:49.504687 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.504668    2580 scope.go:117] "RemoveContainer" containerID="80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"
Sep 12 00:51:49.504974 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:51:49.504955    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4\": container with ID starting with 80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4 not found: ID does not exist" containerID="80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"
Sep 12 00:51:49.505031 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.504982    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4"} err="failed to get container status \"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4\": rpc error: code = NotFound desc = could not find container \"80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4\": container with ID starting with 80c7c9c5f2bef5218fc44c3ff3e40552e189aa0d395d8a061f67c9cd86e71ce4 not found: ID does not exist"
Sep 12 00:51:49.519627 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.519595    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:51:49.523324 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:49.523301    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-64e462-5857b6f555-wds5h"]
Sep 12 00:51:50.374354 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:50.374319    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" path="/var/lib/kubelet/pods/9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9/volumes"
Sep 12 00:51:50.374742 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:50.374713    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" path="/var/lib/kubelet/pods/eba33141-3031-44b7-b8d4-1c62cfc8b26d/volumes"
Sep 12 00:51:54.440909 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:51:54.440874    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:52:09.459903 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.459850    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" podUID="f494e1e9-f464-4b91-a807-38ada37f4adf" containerName="main" containerID="cri-o://57154ec61e3dce2aa980cd148bf71e191f030c0fc903f8c8bd8b11b62305db5f" gracePeriod=45
Sep 12 00:52:09.611820 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.611757    2580 generic.go:356] "Generic (PLEG): container finished" podID="f494e1e9-f464-4b91-a807-38ada37f4adf" containerID="57154ec61e3dce2aa980cd148bf71e191f030c0fc903f8c8bd8b11b62305db5f" exitCode=0
Sep 12 00:52:09.611820 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.611802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" event={"ID":"f494e1e9-f464-4b91-a807-38ada37f4adf","Type":"ContainerDied","Data":"57154ec61e3dce2aa980cd148bf71e191f030c0fc903f8c8bd8b11b62305db5f"}
Sep 12 00:52:09.710577 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.710512    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:52:09.781080 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781046    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781242 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781107    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781242 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svmrj\" (UniqueName: \"kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781242 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781318    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache\") pod \"f494e1e9-f464-4b91-a807-38ada37f4adf\" (UID: \"f494e1e9-f464-4b91-a807-38ada37f4adf\") "
Sep 12 00:52:09.781401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781358    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:09.781401 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781369    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home" (OuterVolumeSpecName: "home") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:09.781629 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781611    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:09.781629 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781625    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:09.781713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.781659    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache" (OuterVolumeSpecName: "model-cache") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:09.783599 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.783570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:52:09.783859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.783831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm" (OuterVolumeSpecName: "dshm") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:09.783969 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.783947    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj" (OuterVolumeSpecName: "kube-api-access-svmrj") pod "f494e1e9-f464-4b91-a807-38ada37f4adf" (UID: "f494e1e9-f464-4b91-a807-38ada37f4adf"). InnerVolumeSpecName "kube-api-access-svmrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:09.882812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.882750    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:09.882812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.882808    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f494e1e9-f464-4b91-a807-38ada37f4adf-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:09.882812 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.882819    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f494e1e9-f464-4b91-a807-38ada37f4adf-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:09.883050 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:09.882828    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svmrj\" (UniqueName: \"kubernetes.io/projected/f494e1e9-f464-4b91-a807-38ada37f4adf-kube-api-access-svmrj\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:52:10.618972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:10.618942    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"
Sep 12 00:52:10.619542 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:10.618988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj" event={"ID":"f494e1e9-f464-4b91-a807-38ada37f4adf","Type":"ContainerDied","Data":"529cb5da32e73415208d509a32c6efdcf95792694d163187daf0509f9de73f49"}
Sep 12 00:52:10.619542 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:10.619038    2580 scope.go:117] "RemoveContainer" containerID="57154ec61e3dce2aa980cd148bf71e191f030c0fc903f8c8bd8b11b62305db5f"
Sep 12 00:52:10.640806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:10.640721    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:52:10.646020 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:10.645996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-64e462/ipp-route-64e462-kserve-6c7bc69bff-sflnj"]
Sep 12 00:52:12.374225 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:12.374189    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f494e1e9-f464-4b91-a807-38ada37f4adf" path="/var/lib/kubelet/pods/f494e1e9-f464-4b91-a807-38ada37f4adf/volumes"
Sep 12 00:52:44.797738 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:52:44.797696    2580 scope.go:117] "RemoveContainer" containerID="aed8c9212b2f01f9ace25e954ab002b3879f8974b98a6cf0597a716adf11463c"
Sep 12 00:53:06.462147 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.462111    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"]
Sep 12 00:53:06.468279 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.468255    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e88680-29819565-vgrrj"]
Sep 12 00:53:06.492424 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.492391    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:53:06.492693 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.492646    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" podUID="251cacb4-c30f-4eed-85f3-c656d5de3271" containerName="payload-processing" containerID="cri-o://2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413" gracePeriod=30
Sep 12 00:53:06.500524 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.500493    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:53:06.500939 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.500906    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" podUID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" containerName="payload-pre-processing" containerID="cri-o://33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea" gracePeriod=30
Sep 12 00:53:06.776713 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.776691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:53:06.779690 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.779668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:53:06.899849 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.899809    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zwnk\" (UniqueName: \"kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk\") pod \"251cacb4-c30f-4eed-85f3-c656d5de3271\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") "
Sep 12 00:53:06.900046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.899859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume\") pod \"251cacb4-c30f-4eed-85f3-c656d5de3271\" (UID: \"251cacb4-c30f-4eed-85f3-c656d5de3271\") "
Sep 12 00:53:06.900046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.899904    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume\") pod \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") "
Sep 12 00:53:06.900046 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.899964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzlwx\" (UniqueName: \"kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx\") pod \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\" (UID: \"ecab940e-efb8-4e5f-beda-c676cb5e3fc4\") "
Sep 12 00:53:06.900348 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.900322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "251cacb4-c30f-4eed-85f3-c656d5de3271" (UID: "251cacb4-c30f-4eed-85f3-c656d5de3271"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:06.900397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.900322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecab940e-efb8-4e5f-beda-c676cb5e3fc4" (UID: "ecab940e-efb8-4e5f-beda-c676cb5e3fc4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:06.902089 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.902064    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk" (OuterVolumeSpecName: "kube-api-access-2zwnk") pod "251cacb4-c30f-4eed-85f3-c656d5de3271" (UID: "251cacb4-c30f-4eed-85f3-c656d5de3271"). InnerVolumeSpecName "kube-api-access-2zwnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:06.902164 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.902092    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx" (OuterVolumeSpecName: "kube-api-access-kzlwx") pod "ecab940e-efb8-4e5f-beda-c676cb5e3fc4" (UID: "ecab940e-efb8-4e5f-beda-c676cb5e3fc4"). InnerVolumeSpecName "kube-api-access-kzlwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:06.929624 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.929586    2580 generic.go:356] "Generic (PLEG): container finished" podID="251cacb4-c30f-4eed-85f3-c656d5de3271" containerID="2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413" exitCode=0
Sep 12 00:53:06.929819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.929636    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"
Sep 12 00:53:06.929819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.929675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" event={"ID":"251cacb4-c30f-4eed-85f3-c656d5de3271","Type":"ContainerDied","Data":"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"}
Sep 12 00:53:06.929819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.929727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm" event={"ID":"251cacb4-c30f-4eed-85f3-c656d5de3271","Type":"ContainerDied","Data":"08616722bd884d9d3a9ff9b5f14916dc9c7063238a2a1e8a6a1ff3a95fa1f0a4"}
Sep 12 00:53:06.929819 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.929793    2580 scope.go:117] "RemoveContainer" containerID="2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"
Sep 12 00:53:06.931477 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.931451    2580 generic.go:356] "Generic (PLEG): container finished" podID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" containerID="33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea" exitCode=0
Sep 12 00:53:06.931576 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.931493    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"
Sep 12 00:53:06.931576 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.931501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" event={"ID":"ecab940e-efb8-4e5f-beda-c676cb5e3fc4","Type":"ContainerDied","Data":"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"}
Sep 12 00:53:06.931576 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.931551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf" event={"ID":"ecab940e-efb8-4e5f-beda-c676cb5e3fc4","Type":"ContainerDied","Data":"216ee919b50cdcf761ab0d4cc749a3eb5ce0fecafd66766274b80be88e912dc6"}
Sep 12 00:53:06.939631 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.939609    2580 scope.go:117] "RemoveContainer" containerID="2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"
Sep 12 00:53:06.939954 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:53:06.939933    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413\": container with ID starting with 2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413 not found: ID does not exist" containerID="2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"
Sep 12 00:53:06.940011 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.939962    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413"} err="failed to get container status \"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413\": rpc error: code = NotFound desc = could not find container \"2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413\": container with ID starting with 2ecd097dfc129bb8611dd744cabd4e2b9d59b2c7ec576f238edeb0cde32f2413 not found: ID does not exist"
Sep 12 00:53:06.940011 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.939978    2580 scope.go:117] "RemoveContainer" containerID="33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"
Sep 12 00:53:06.948385 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.948366    2580 scope.go:117] "RemoveContainer" containerID="33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"
Sep 12 00:53:06.948641 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:53:06.948620    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea\": container with ID starting with 33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea not found: ID does not exist" containerID="33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"
Sep 12 00:53:06.948702 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.948652    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea"} err="failed to get container status \"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea\": rpc error: code = NotFound desc = could not find container \"33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea\": container with ID starting with 33f91db1b53ef9709827299eec6f45dd535212ccfcd23dcbe26d8912959f06ea not found: ID does not exist"
Sep 12 00:53:06.955807 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.955784    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:53:06.960093 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.960074    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e88680-6b4bfd4874-2bnwf"]
Sep 12 00:53:06.969620 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.969598    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:53:06.975370 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:06.975349    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e88680-7df4f5564f-9d4gm"]
Sep 12 00:53:07.001125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:07.001060    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:07.001125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:07.001083    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzlwx\" (UniqueName: \"kubernetes.io/projected/ecab940e-efb8-4e5f-beda-c676cb5e3fc4-kube-api-access-kzlwx\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:07.001125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:07.001097    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zwnk\" (UniqueName: \"kubernetes.io/projected/251cacb4-c30f-4eed-85f3-c656d5de3271-kube-api-access-2zwnk\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:07.001125 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:07.001110    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251cacb4-c30f-4eed-85f3-c656d5de3271-plugins-config-volume\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:08.373925 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:08.373891    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04542f81-532d-4071-9af2-884d58052712" path="/var/lib/kubelet/pods/04542f81-532d-4071-9af2-884d58052712/volumes"
Sep 12 00:53:08.374303 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:08.374236    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="251cacb4-c30f-4eed-85f3-c656d5de3271" path="/var/lib/kubelet/pods/251cacb4-c30f-4eed-85f3-c656d5de3271/volumes"
Sep 12 00:53:08.374554 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:08.374540    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" path="/var/lib/kubelet/pods/ecab940e-efb8-4e5f-beda-c676cb5e3fc4/volumes"
Sep 12 00:53:24.157602 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.157560    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:53:24.158033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.157825    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" podUID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" containerName="maas-api" containerID="cri-o://caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8" gracePeriod=30
Sep 12 00:53:24.189500 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.189463    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"]
Sep 12 00:53:24.192917 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.192890    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e37bfb-29819565-6gsxg"]
Sep 12 00:53:24.376249 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.376212    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90c0ad9c-b5bc-4952-a690-2adb79dd2634" path="/var/lib/kubelet/pods/90c0ad9c-b5bc-4952-a690-2adb79dd2634/volumes"
Sep 12 00:53:24.404205 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.404182    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:53:24.566143 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.566110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") pod \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") "
Sep 12 00:53:24.566350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.566197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca\") pod \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") "
Sep 12 00:53:24.566350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.566269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs\") pod \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") "
Sep 12 00:53:24.566350 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.566306    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv9md\" (UniqueName: \"kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md\") pod \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\" (UID: \"139c5f59-0f7d-4615-b2ce-2462c42d2dac\") "
Sep 12 00:53:24.566654 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.566615    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "139c5f59-0f7d-4615-b2ce-2462c42d2dac" (UID: "139c5f59-0f7d-4615-b2ce-2462c42d2dac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:24.568406 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.568374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "139c5f59-0f7d-4615-b2ce-2462c42d2dac" (UID: "139c5f59-0f7d-4615-b2ce-2462c42d2dac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:24.568512 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.568431    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "139c5f59-0f7d-4615-b2ce-2462c42d2dac" (UID: "139c5f59-0f7d-4615-b2ce-2462c42d2dac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:24.568512 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.568501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md" (OuterVolumeSpecName: "kube-api-access-wv9md") pod "139c5f59-0f7d-4615-b2ce-2462c42d2dac" (UID: "139c5f59-0f7d-4615-b2ce-2462c42d2dac"). InnerVolumeSpecName "kube-api-access-wv9md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:24.666994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.666956    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/139c5f59-0f7d-4615-b2ce-2462c42d2dac-openshift-service-ca\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.666994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.666989    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-metrics-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.666994 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.666999    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv9md\" (UniqueName: \"kubernetes.io/projected/139c5f59-0f7d-4615-b2ce-2462c42d2dac-kube-api-access-wv9md\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.667224 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:24.667007    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/139c5f59-0f7d-4615-b2ce-2462c42d2dac-maas-api-tls\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:25.033679 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.033643    2580 generic.go:356] "Generic (PLEG): container finished" podID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" containerID="caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8" exitCode=0
Sep 12 00:53:25.033874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.033714    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"
Sep 12 00:53:25.033874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.033728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" event={"ID":"139c5f59-0f7d-4615-b2ce-2462c42d2dac","Type":"ContainerDied","Data":"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"}
Sep 12 00:53:25.033874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.033835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht" event={"ID":"139c5f59-0f7d-4615-b2ce-2462c42d2dac","Type":"ContainerDied","Data":"2d40d74068b9aeb232321eddff10ab35f54d55af4ae222cb2396f34d68eb3827"}
Sep 12 00:53:25.033874 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.033861    2580 scope.go:117] "RemoveContainer" containerID="caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"
Sep 12 00:53:25.043539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.043516    2580 scope.go:117] "RemoveContainer" containerID="caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"
Sep 12 00:53:25.043827 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:53:25.043802    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8\": container with ID starting with caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8 not found: ID does not exist" containerID="caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"
Sep 12 00:53:25.043916 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.043834    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8"} err="failed to get container status \"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8\": rpc error: code = NotFound desc = could not find container \"caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8\": container with ID starting with caaaabe47c06fabe1f81a963a3db61dcf2cc957f7e04725e9ccb8e94064237d8 not found: ID does not exist"
Sep 12 00:53:25.056024 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.056003    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:53:25.061392 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:25.061368    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e37bfb-67546fb8c-ng2ht"]
Sep 12 00:53:26.373521 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:26.373489    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" path="/var/lib/kubelet/pods/139c5f59-0f7d-4615-b2ce-2462c42d2dac/volumes"
Sep 12 00:53:31.211551 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:31.211522    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:53:31.211954 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:31.211752    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" containerID="cri-o://52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6" gracePeriod=30
Sep 12 00:53:35.121339 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:35.121303    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:53:35.121725 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:35.121374    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:53:36.456734 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.456712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:53:36.477054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.476992    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477054 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477479 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph8v5\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477479 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477519    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:36.477618 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477733 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477628    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477733 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477733 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert\") pod \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\" (UID: \"099e16c8-2879-4b8c-a3a6-31e58f6cad28\") "
Sep 12 00:53:36.477916 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.477894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:36.478024 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.478006    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.478078 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.478028    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-credential-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.478484 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.478271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:36.478702 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.478648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data" (OuterVolumeSpecName: "istio-data") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:36.478889 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.478666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:36.481068 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.481038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token" (OuterVolumeSpecName: "istio-token") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:36.481165 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.481143    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:36.481219 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.481198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:53:36.481516 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.481490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5" (OuterVolumeSpecName: "kube-api-access-ph8v5") pod "099e16c8-2879-4b8c-a3a6-31e58f6cad28" (UID: "099e16c8-2879-4b8c-a3a6-31e58f6cad28"). InnerVolumeSpecName "kube-api-access-ph8v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:36.578525 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578488    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-token\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578525 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578518    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph8v5\" (UniqueName: \"kubernetes.io/projected/099e16c8-2879-4b8c-a3a6-31e58f6cad28-kube-api-access-ph8v5\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578525 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578529    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-envoy\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578538    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-podinfo\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578546    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-workload-socket\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578555    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istiod-ca-cert\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:36.578761 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:36.578563    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099e16c8-2879-4b8c-a3a6-31e58f6cad28-istio-data\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:53:37.101346 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.101309    2580 generic.go:356] "Generic (PLEG): container finished" podID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerID="52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6" exitCode=0
Sep 12 00:53:37.101531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.101356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"
Sep 12 00:53:37.101531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.101367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerDied","Data":"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"}
Sep 12 00:53:37.101531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.101427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7" event={"ID":"099e16c8-2879-4b8c-a3a6-31e58f6cad28","Type":"ContainerDied","Data":"811d351f82b46f34ee1f9279b20e130f5c2ebc0be6d41e5387101e535e889611"}
Sep 12 00:53:37.101531 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.101449    2580 scope.go:117] "RemoveContainer" containerID="52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"
Sep 12 00:53:37.111157 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.111139    2580 scope.go:117] "RemoveContainer" containerID="d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"
Sep 12 00:53:37.121462 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.121432    2580 scope.go:117] "RemoveContainer" containerID="52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"
Sep 12 00:53:37.121829 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:53:37.121807    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6\": container with ID starting with 52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6 not found: ID does not exist" containerID="52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"
Sep 12 00:53:37.121922 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.121841    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6"} err="failed to get container status \"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6\": rpc error: code = NotFound desc = could not find container \"52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6\": container with ID starting with 52a846bab6fbd3705332a513e5d26c59b5978f281c9c72a0b327c42bce0524a6 not found: ID does not exist"
Sep 12 00:53:37.121922 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.121864    2580 scope.go:117] "RemoveContainer" containerID="d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"
Sep 12 00:53:37.122136 ip-10-0-134-179 kubenswrapper[2580]: E0912 00:53:37.122117    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d\": container with ID starting with d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d not found: ID does not exist" containerID="d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"
Sep 12 00:53:37.122184 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.122143    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d"} err="failed to get container status \"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d\": rpc error: code = NotFound desc = could not find container \"d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d\": container with ID starting with d6df22268578dd55baad03573e5f20773e2ad422df33782c829f83ef04570f1d not found: ID does not exist"
Sep 12 00:53:37.123222 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.123202    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:53:37.126838 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:37.126818    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e37bfb-openshift-default-b7bf588f6-jlzn7"]
Sep 12 00:53:38.373899 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:38.373865    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" path="/var/lib/kubelet/pods/099e16c8-2879-4b8c-a3a6-31e58f6cad28/volumes"
Sep 12 00:53:44.827924 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:44.827894    2580 scope.go:117] "RemoveContainer" containerID="210ca85e18231e79b9447d436ddfd5491ed0305e12ee2383db5357308270b7f2"
Sep 12 00:53:44.836460 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:53:44.836432    2580 scope.go:117] "RemoveContainer" containerID="16bce250146ef8def4ab793bf09f2d010540444f475abcd707e9b7bdfa6847b5"
Sep 12 00:57:44.896045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:44.895941    2580 scope.go:117] "RemoveContainer" containerID="b60cc0125b47418f29c1d4b39bd0d36774f50e5d22b4d53fe127a14a0bc7efc0"
Sep 12 00:57:44.904468 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:44.904439    2580 scope.go:117] "RemoveContainer" containerID="8470a879a773a521d607af5aa24c0f6fc52257be1e051c88c91a89b34e9040ac"
Sep 12 00:57:49.121248 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121215    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121545    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121557    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121569    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" containerName="payload-pre-processing"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121575    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" containerName="payload-pre-processing"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121581    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121586    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121597    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" containerName="payload-processing"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121602    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" containerName="payload-processing"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121610    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" containerName="revoke-keys"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121616    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" containerName="revoke-keys"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121623    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f494e1e9-f464-4b91-a807-38ada37f4adf" containerName="main"
Sep 12 00:57:49.121622 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121628    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f494e1e9-f464-4b91-a807-38ada37f4adf" containerName="main"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121638    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="251cacb4-c30f-4eed-85f3-c656d5de3271" containerName="payload-processing"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121643    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="251cacb4-c30f-4eed-85f3-c656d5de3271" containerName="payload-processing"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121651    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" containerName="maas-api"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121655    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" containerName="maas-api"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121702    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba33141-3031-44b7-b8d4-1c62cfc8b26d" containerName="payload-processing"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121713    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121719    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f494e1e9-f464-4b91-a807-38ada37f4adf" containerName="main"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121725    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecab940e-efb8-4e5f-beda-c676cb5e3fc4" containerName="payload-pre-processing"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121731    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="139c5f59-0f7d-4615-b2ce-2462c42d2dac" containerName="maas-api"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121737    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4f6727-37e5-4f6c-8c92-5ec042bdc0b9" containerName="revoke-keys"
Sep 12 00:57:49.122015 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.121745    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="251cacb4-c30f-4eed-85f3-c656d5de3271" containerName="payload-processing"
Sep 12 00:57:49.125972 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.125952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.128991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.128970    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8b139d27-kserve-self-signed-certs\""
Sep 12 00:57:49.140250 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.140229    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:57:49.237671 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.237888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.237888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkv5b\" (UniqueName: \"kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.237888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.237888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.237888 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.237830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338714 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkv5b\" (UniqueName: \"kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338714 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.338991 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.338901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.339271 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.339236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.339271 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.339264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.339383 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.339256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.340973 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.340955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.341268 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.341250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.346162 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.346137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkv5b\" (UniqueName: \"kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b\") pod \"e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") " pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.435295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.435264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:49.707944 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:49.707915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:57:49.709265 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:57:49.709242    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcdfe862_6c05_4285_b68b_07e522212c63.slice/crio-b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d WatchSource:0}: Error finding container b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d: Status 404 returned error can't find the container with id b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d
Sep 12 00:57:50.404316 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:50.404282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" event={"ID":"dcdfe862-6c05-4285-b68b-07e522212c63","Type":"ContainerStarted","Data":"a101a691ababeb5800df8c7a34b702d3df48e072cc75a0258f3522397e39bb5d"}
Sep 12 00:57:50.404706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:50.404328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" event={"ID":"dcdfe862-6c05-4285-b68b-07e522212c63","Type":"ContainerStarted","Data":"b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d"}
Sep 12 00:57:50.423088 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:50.423034    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" podStartSLOduration=1.423018044 podStartE2EDuration="1.423018044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:57:50.420726542 +0000 UTC m=+2466.554716769" watchObservedRunningTime="2026-09-12 00:57:50.423018044 +0000 UTC m=+2466.557008270"
Sep 12 00:57:59.435599 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:59.435555    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:59.436116 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:59.435620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:59.448362 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:59.448335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:57:59.460779 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:57:59.460742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:58:15.344946 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:15.344908    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:58:30.363623 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.363554    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" podUID="dcdfe862-6c05-4285-b68b-07e522212c63" containerName="main" containerID="cri-o://a101a691ababeb5800df8c7a34b702d3df48e072cc75a0258f3522397e39bb5d" gracePeriod=45
Sep 12 00:58:30.615832 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.615750    2580 generic.go:356] "Generic (PLEG): container finished" podID="dcdfe862-6c05-4285-b68b-07e522212c63" containerID="a101a691ababeb5800df8c7a34b702d3df48e072cc75a0258f3522397e39bb5d" exitCode=0
Sep 12 00:58:30.615971 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.615825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" event={"ID":"dcdfe862-6c05-4285-b68b-07e522212c63","Type":"ContainerDied","Data":"a101a691ababeb5800df8c7a34b702d3df48e072cc75a0258f3522397e39bb5d"}
Sep 12 00:58:30.615971 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.615866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm" event={"ID":"dcdfe862-6c05-4285-b68b-07e522212c63","Type":"ContainerDied","Data":"b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d"}
Sep 12 00:58:30.615971 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.615879    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b76170bdff9bd69e3c75c5cc1044ef1fae45c6e26d9a5e708276c614ebfb9d0d"
Sep 12 00:58:30.616655 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.616641    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:58:30.703137 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703099    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703156    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703184    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703214    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703274    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkv5b\" (UniqueName: \"kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703340    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home\") pod \"dcdfe862-6c05-4285-b68b-07e522212c63\" (UID: \"dcdfe862-6c05-4285-b68b-07e522212c63\") "
Sep 12 00:58:30.703533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:58:30.703533 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache" (OuterVolumeSpecName: "model-cache") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:58:30.703704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703637    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home" (OuterVolumeSpecName: "home") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:58:30.703704 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703696    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-tmp-dir\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:30.703814 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.703712    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-model-cache\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:30.705563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.705539    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm" (OuterVolumeSpecName: "dshm") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:58:30.705563 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.705542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b" (OuterVolumeSpecName: "kube-api-access-pkv5b") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "kube-api-access-pkv5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:58:30.705692 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.705584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dcdfe862-6c05-4285-b68b-07e522212c63" (UID: "dcdfe862-6c05-4285-b68b-07e522212c63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:58:30.805033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.804998    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-dshm\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:30.805033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.805028    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcdfe862-6c05-4285-b68b-07e522212c63-tls-certs\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:30.805033 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.805038    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkv5b\" (UniqueName: \"kubernetes.io/projected/dcdfe862-6c05-4285-b68b-07e522212c63-kube-api-access-pkv5b\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:30.805271 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:30.805048    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcdfe862-6c05-4285-b68b-07e522212c63-home\") on node \"ip-10-0-134-179.ec2.internal\" DevicePath \"\""
Sep 12 00:58:31.621455 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:31.621414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"
Sep 12 00:58:31.641471 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:31.641439    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:58:31.645045 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:31.645019    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8b139d27-kserve-7dc65957c9-6s6fm"]
Sep 12 00:58:32.374456 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:32.374421    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcdfe862-6c05-4285-b68b-07e522212c63" path="/var/lib/kubelet/pods/dcdfe862-6c05-4285-b68b-07e522212c63/volumes"
Sep 12 00:58:41.868825 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.868790    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"]
Sep 12 00:58:41.869210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.869177    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcdfe862-6c05-4285-b68b-07e522212c63" containerName="main"
Sep 12 00:58:41.869210 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.869188    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcdfe862-6c05-4285-b68b-07e522212c63" containerName="main"
Sep 12 00:58:41.869295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.869253    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcdfe862-6c05-4285-b68b-07e522212c63" containerName="main"
Sep 12 00:58:41.869295 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.869260    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="099e16c8-2879-4b8c-a3a6-31e58f6cad28" containerName="istio-proxy"
Sep 12 00:58:41.872515 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.872492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:41.874876 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.874852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 12 00:58:41.875013 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.874898    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 00:58:41.875672 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.875653    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 12 00:58:41.875672 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.875668    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f4scw\""
Sep 12 00:58:41.875834 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.875719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 12 00:58:41.880755 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:41.880732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"]
Sep 12 00:58:42.011055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.011015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa503aa6-a06a-4997-a409-e1eb881b9772-openshift-service-ca\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.011055 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.011061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8dsb\" (UniqueName: \"kubernetes.io/projected/fa503aa6-a06a-4997-a409-e1eb881b9772-kube-api-access-f8dsb\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.011276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.011089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-metrics-certs\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.011276 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.011130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-maas-api-tls\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.112209 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.112172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa503aa6-a06a-4997-a409-e1eb881b9772-openshift-service-ca\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.112396 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.112215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8dsb\" (UniqueName: \"kubernetes.io/projected/fa503aa6-a06a-4997-a409-e1eb881b9772-kube-api-access-f8dsb\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.112396 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.112253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-metrics-certs\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.112396 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.112292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-maas-api-tls\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.112859 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.112837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa503aa6-a06a-4997-a409-e1eb881b9772-openshift-service-ca\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.114654 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.114628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-maas-api-tls\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.114813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.114792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa503aa6-a06a-4997-a409-e1eb881b9772-metrics-certs\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.120843 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.120791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8dsb\" (UniqueName: \"kubernetes.io/projected/fa503aa6-a06a-4997-a409-e1eb881b9772-kube-api-access-f8dsb\") pod \"maas-api-6bb595b6d8-6dtdq\" (UID: \"fa503aa6-a06a-4997-a409-e1eb881b9772\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.184722 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.184683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:42.460553 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.460522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"]
Sep 12 00:58:42.462688 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:58:42.462661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa503aa6_a06a_4997_a409_e1eb881b9772.slice/crio-98ae6cab6441b3c4e7b4434c18dea5d71205a4e0a2099f5cd0d697dbb190b1ac WatchSource:0}: Error finding container 98ae6cab6441b3c4e7b4434c18dea5d71205a4e0a2099f5cd0d697dbb190b1ac: Status 404 returned error can't find the container with id 98ae6cab6441b3c4e7b4434c18dea5d71205a4e0a2099f5cd0d697dbb190b1ac
Sep 12 00:58:42.464487 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.464470    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:58:42.681135 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:42.681099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq" event={"ID":"fa503aa6-a06a-4997-a409-e1eb881b9772","Type":"ContainerStarted","Data":"98ae6cab6441b3c4e7b4434c18dea5d71205a4e0a2099f5cd0d697dbb190b1ac"}
Sep 12 00:58:43.688908 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:43.688873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq" event={"ID":"fa503aa6-a06a-4997-a409-e1eb881b9772","Type":"ContainerStarted","Data":"76bed1877aff74e65efd58d84802ea8bade218f4c7d82fbc41c090df198d1beb"}
Sep 12 00:58:43.689314 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:43.689004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:58:43.706539 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:43.706468    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq" podStartSLOduration=2.388707443 podStartE2EDuration="2.706454218s" totalImagesPullingTime="317.746775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:58:43.705362656 +0000 UTC m=+2519.839352882" watchObservedRunningTime="2026-09-12 00:58:43.706454218 +0000 UTC m=+2519.840444467"
Sep 12 00:58:49.703044 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:58:49.703013    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-6dtdq"
Sep 12 00:59:04.601724 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:04.601685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595995c945-f6lzz_e94ed612-a352-4b76-9a81-a7ba95a00792/authorino/0.log"
Sep 12 00:59:10.625652 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:10.625617    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-m72zr_1e60429e-5b41-4503-b185-ed999848015e/manager/0.log"
Sep 12 00:59:10.987458 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:10.987430    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-qpsx4_2dd846bb-a92d-4c58-bde8-a96595141003/copy-manifests/0.log"
Sep 12 00:59:10.994177 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:10.994152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-qpsx4_2dd846bb-a92d-4c58-bde8-a96595141003/manager/0.log"
Sep 12 00:59:11.354861 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:11.354758    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bb595b6d8-6dtdq_fa503aa6-a06a-4997-a409-e1eb881b9772/maas-api/0.log"
Sep 12 00:59:11.586514 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:11.586484    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819565-lchr7_7b60b1ca-ec9e-49a8-88a0-5451e236aebf/cleanup/0.log"
Sep 12 00:59:12.515034 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:12.515005    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595995c945-f6lzz_e94ed612-a352-4b76-9a81-a7ba95a00792/authorino/0.log"
Sep 12 00:59:13.623273 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:13.623231    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm_da45d3d9-52c8-4978-8e87-2b435cb10994/istio-proxy/0.log"
Sep 12 00:59:13.993643 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:13.993612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-g78ps_501c7316-0153-4431-b6ce-0160608b9a0c/kube-auth-proxy/0.log"
Sep 12 00:59:15.082420 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:15.082388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-7p58g_9908bc37-4700-45e5-8786-724351bee1d9/main/0.log"
Sep 12 00:59:15.324468 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:15.324427    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fm6qq5_509f4d03-d9fc-4a53-80a9-9c396aa57acb/main/0.log"
Sep 12 00:59:15.446397 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:15.446360    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8hqxq_ae7b9845-bb62-471f-a1b3-8924846546d0/main/0.log"
Sep 12 00:59:23.674435 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:23.674387    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-72whv_f79f9dca-61f6-45f2-b1e5-248e521d599f/global-pull-secret-syncer/0.log"
Sep 12 00:59:23.827465 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:23.827435    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lxhxh_52f1aae7-285c-4898-be26-2e26dc23bcc2/konnectivity-agent/0.log"
Sep 12 00:59:23.880155 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:23.880120    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-179.ec2.internal_9f84cc7b6dd936cd66de2a1a58b0376d/haproxy/0.log"
Sep 12 00:59:28.345864 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:28.345833    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595995c945-f6lzz_e94ed612-a352-4b76-9a81-a7ba95a00792/authorino/0.log"
Sep 12 00:59:30.396822 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:30.396761    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-55xws_6019a323-7283-4391-916d-10bf862e0cff/cluster-monitoring-operator/0.log"
Sep 12 00:59:30.647338 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:30.647259    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qpwgx_83ad8709-acd6-45d4-953c-1856c229ce21/node-exporter/0.log"
Sep 12 00:59:30.667897 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:30.667868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qpwgx_83ad8709-acd6-45d4-953c-1856c229ce21/kube-rbac-proxy/0.log"
Sep 12 00:59:30.690438 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:30.690407    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qpwgx_83ad8709-acd6-45d4-953c-1856c229ce21/init-textfile/0.log"
Sep 12 00:59:31.101921 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.101888    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kdv8t_72d6568b-7488-499f-b4be-e177b12de27c/prometheus-operator/0.log"
Sep 12 00:59:31.127059 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.126998    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kdv8t_72d6568b-7488-499f-b4be-e177b12de27c/kube-rbac-proxy/0.log"
Sep 12 00:59:31.153111 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.153079    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-knhxf_7e1cdeb9-aaf7-4d47-8ff7-7d742a72cfe6/prometheus-operator-admission-webhook/0.log"
Sep 12 00:59:31.433639 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.433615    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/thanos-query/0.log"
Sep 12 00:59:31.472634 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.472604    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/kube-rbac-proxy-web/0.log"
Sep 12 00:59:31.504081 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.504045    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/kube-rbac-proxy/0.log"
Sep 12 00:59:31.524927 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.524902    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/prom-label-proxy/0.log"
Sep 12 00:59:31.545781 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.545738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/kube-rbac-proxy-rules/0.log"
Sep 12 00:59:31.566368 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:31.566339    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f89f6dc95-57jlh_c9b97280-292c-486c-9743-d59408327a15/kube-rbac-proxy-metrics/0.log"
Sep 12 00:59:33.783161 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.783122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"]
Sep 12 00:59:33.786813 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.786790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.789546 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.789522    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvbw5\"/\"openshift-service-ca.crt\""
Sep 12 00:59:33.789720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.789704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wvbw5\"/\"default-dockercfg-wwsd6\""
Sep 12 00:59:33.790692 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.790662    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvbw5\"/\"kube-root-ca.crt\""
Sep 12 00:59:33.796647 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.796625    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"]
Sep 12 00:59:33.887862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.883084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-proc\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.887862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.883141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-lib-modules\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.887862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.883206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gh7l\" (UniqueName: \"kubernetes.io/projected/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-kube-api-access-9gh7l\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.887862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.883245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-sys\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.887862 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.883282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-podres\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984473 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-podres\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984645 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-proc\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984645 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-lib-modules\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984645 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gh7l\" (UniqueName: \"kubernetes.io/projected/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-kube-api-access-9gh7l\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984645 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-podres\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-sys\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-proc\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-sys\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.984806 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.984733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-lib-modules\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:33.994442 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:33.994418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gh7l\" (UniqueName: \"kubernetes.io/projected/258ce46d-7c59-4b6e-8b84-f8bc18ef0998-kube-api-access-9gh7l\") pod \"perf-node-gather-daemonset-gd48z\" (UID: \"258ce46d-7c59-4b6e-8b84-f8bc18ef0998\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:34.098017 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.097940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:34.175682 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.175658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f7c784bcf-w5sjw_13f345ef-1e12-402d-9577-38b71e30e676/console/0.log"
Sep 12 00:59:34.373747 ip-10-0-134-179 kubenswrapper[2580]: W0912 00:59:34.373717    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod258ce46d_7c59_4b6e_8b84_f8bc18ef0998.slice/crio-4b9b6760b6f657a9b6fdfd61f6a53a01f2664a5ffc60a05b4c8a4d3c9e1f8af4 WatchSource:0}: Error finding container 4b9b6760b6f657a9b6fdfd61f6a53a01f2664a5ffc60a05b4c8a4d3c9e1f8af4: Status 404 returned error can't find the container with id 4b9b6760b6f657a9b6fdfd61f6a53a01f2664a5ffc60a05b4c8a4d3c9e1f8af4
Sep 12 00:59:34.374857 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.374822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"]
Sep 12 00:59:34.961292 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.961251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z" event={"ID":"258ce46d-7c59-4b6e-8b84-f8bc18ef0998","Type":"ContainerStarted","Data":"3c46db1d66bdd2e6a0d610836cc294156374405b689e2d174f5750bac35e4a7d"}
Sep 12 00:59:34.961720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.961307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z" event={"ID":"258ce46d-7c59-4b6e-8b84-f8bc18ef0998","Type":"ContainerStarted","Data":"4b9b6760b6f657a9b6fdfd61f6a53a01f2664a5ffc60a05b4c8a4d3c9e1f8af4"}
Sep 12 00:59:34.961720 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.961331    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:34.981255 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:34.981202    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z" podStartSLOduration=1.981184303 podStartE2EDuration="1.981184303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:59:34.979202526 +0000 UTC m=+2571.113192752" watchObservedRunningTime="2026-09-12 00:59:34.981184303 +0000 UTC m=+2571.115174528"
Sep 12 00:59:36.201658 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:36.201622    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx595_3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6/dns/0.log"
Sep 12 00:59:36.222858 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:36.222825    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx595_3ebc64a9-7fc9-40dc-ba57-5c9d62a8a3d6/kube-rbac-proxy/0.log"
Sep 12 00:59:36.246234 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:36.246202    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5vv4d_21d75218-5ef8-477b-930e-0f3cac2806af/dns-node-resolver/0.log"
Sep 12 00:59:36.868706 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:36.868676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jnf7d_6e50127b-bff1-431e-aa20-3240d34fbc0b/node-ca/0.log"
Sep 12 00:59:37.812453 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:37.812413    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm_da45d3d9-52c8-4978-8e87-2b435cb10994/istio-proxy/0.log"
Sep 12 00:59:38.238998 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:38.238967    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-g78ps_501c7316-0153-4431-b6ce-0160608b9a0c/kube-auth-proxy/0.log"
Sep 12 00:59:39.295315 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:39.295282    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrrp5_7b3ab479-faf4-46d8-af56-b8295bb9c41d/serve-healthcheck-canary/0.log"
Sep 12 00:59:39.936815 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:39.936788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-svbxq_dcf1f4a7-b086-434a-97a4-a03af21aef51/kube-rbac-proxy/0.log"
Sep 12 00:59:39.967488 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:39.967456    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-svbxq_dcf1f4a7-b086-434a-97a4-a03af21aef51/exporter/0.log"
Sep 12 00:59:39.986687 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:39.986658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-svbxq_dcf1f4a7-b086-434a-97a4-a03af21aef51/extractor/0.log"
Sep 12 00:59:40.976059 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:40.976032    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-gd48z"
Sep 12 00:59:42.482005 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:42.481973    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-m72zr_1e60429e-5b41-4503-b185-ed999848015e/manager/0.log"
Sep 12 00:59:42.674727 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:42.674696    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-qpsx4_2dd846bb-a92d-4c58-bde8-a96595141003/manager/0.log"
Sep 12 00:59:42.696504 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:42.696475    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-qpsx4_2dd846bb-a92d-4c58-bde8-a96595141003/copy-manifests/0.log"
Sep 12 00:59:44.083439 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:44.083408    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm_da45d3d9-52c8-4978-8e87-2b435cb10994/istio-proxy/0.log"
Sep 12 00:59:44.178828 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:44.178797    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b5cf77b9-vxqkk_15a1e0cb-4c11-41cd-884d-0c1b9535726d/manager/0.log"
Sep 12 00:59:44.490834 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:44.490740    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-g78ps_501c7316-0153-4431-b6ce-0160608b9a0c/kube-auth-proxy/0.log"
Sep 12 00:59:46.092369 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:46.092338    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-tmqfm_da45d3d9-52c8-4978-8e87-2b435cb10994/istio-proxy/0.log"
Sep 12 00:59:46.511710 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:46.511680    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-g78ps_501c7316-0153-4431-b6ce-0160608b9a0c/kube-auth-proxy/0.log"
Sep 12 00:59:50.950032 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:50.950000    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vmbg5_fe313196-f058-4a7a-9de7-6307e39a7d4a/kube-multus-additional-cni-plugins/0.log"
Sep 12 00:59:50.970356 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:50.970326    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vmbg5_fe313196-f058-4a7a-9de7-6307e39a7d4a/egress-router-binary-copy/0.log"
Sep 12 00:59:50.991365 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:50.991341    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vmbg5_fe313196-f058-4a7a-9de7-6307e39a7d4a/cni-plugins/0.log"
Sep 12 00:59:51.012464 ip-10-0-134-179 kubenswrapper[2580]: I0912 00:59:51.012430    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vmbg5_fe313196-f058-4a7a-9de7-6307e39a7d4a/bond-cni-plugin/0.log"