Sep 16 16:34:31.170212 ip-10-0-132-17 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 16:34:31.170241 ip-10-0-132-17 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 16:34:31.170251 ip-10-0-132-17 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 16:34:31.170619 ip-10-0-132-17 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 16:34:41.385203 ip-10-0-132-17 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 16:34:41.385220 ip-10-0-132-17 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot fea442ad0fa24f8a990b8d74bc210fcd --
Sep 16 16:36:22.785634 ip-10-0-132-17 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 16:36:23.229933 ip-10-0-132-17 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:23.229933 ip-10-0-132-17 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 16:36:23.229933 ip-10-0-132-17 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:23.229933 ip-10-0-132-17 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 16:36:23.229933 ip-10-0-132-17 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:23.232473 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.232404    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 16:36:23.238117 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238097    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.238117 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238114    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.238117 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238118    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.238117 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238121    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238125    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238128    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238130    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238133    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238136    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238140    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238143    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238146    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238149    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238153    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238155    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238158    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238161    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238164    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238166    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238169    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238171    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238174    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238177    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.238273 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238181    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238185    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238188    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238191    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238194    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238197    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238207    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238211    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238213    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238216    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238218    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238221    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238223    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238226    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238229    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238232    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238235    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238238    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238240    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.238769 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238243    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238245    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238248    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238251    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238253    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238257    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238261    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238264    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238267    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238270    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238273    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238276    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238280    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238283    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238285    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238289    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238293    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238297    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238299    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.239247 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238302    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238304    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238307    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238309    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238312    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238314    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238317    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238320    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238323    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238326    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238329    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238332    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238335    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238338    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238340    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238343    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238345    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238348    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238351    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238354    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.239728 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238357    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238359    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238362    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238365    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238368    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238371    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238373    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238376    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238378    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238392    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238395    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238397    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238400    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238402    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238405    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238407    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238410    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.238413    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239096    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239102    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.240255 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239106    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239109    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239112    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239115    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239118    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239121    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239123    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239126    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239129    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239132    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239135    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239137    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239140    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239143    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239145    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239148    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239150    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239153    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239155    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239158    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.240753 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239161    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239163    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239165    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239168    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239171    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239173    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239176    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239178    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239183    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239186    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239189    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239192    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239195    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239198    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239201    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239204    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239206    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239210    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239212    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.241253 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239216    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239220    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239223    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239226    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239228    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239231    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239233    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239236    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239238    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239242    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239246    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239249    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239252    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239254    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239257    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239259    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239262    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239265    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239268    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.241729 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239271    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239273    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239276    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239278    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239281    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239283    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239286    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239289    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239292    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239295    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239297    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239300    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239303    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239305    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239308    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239310    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239313    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239315    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239318    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239320    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.242196 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239323    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239325    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239328    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239331    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239333    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239336    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239338    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239341    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239344    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239346    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239348    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239351    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239353    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239356    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239358    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239361    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239363    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239366    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239369    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239770    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.242691 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239775    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239778    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239781    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239785    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239787    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239790    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239793    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239796    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239799    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239801    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239804    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239806    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239809    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239812    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239814    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239817    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239821    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239824    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239826    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.243184 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239829    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239833    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239837    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239841    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239844    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239847    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239850    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239853    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239856    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239859    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239861    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239866    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239869    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239872    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239874    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239877    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239879    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239882    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239884    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.243661 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239887    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239889    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239892    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239894    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239897    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239899    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239902    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239905    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239907    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239910    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239913    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239915    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239918    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239920    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239923    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239925    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239928    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239930    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239933    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239935    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.244137 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239938    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239942    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239945    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239947    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239951    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239954    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239956    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239959    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239961    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239964    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239966    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239969    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239971    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239974    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239977    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239979    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239982    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239984    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239989    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239992    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.244669 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239994    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239997    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.239999    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240010    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240013    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240015    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240018    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240020    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240024    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240027    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240030    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240033    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240036    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240041    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240044    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240047    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240051    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240053    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240056    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240059    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.245166 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240140    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240178    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240184    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240188    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240192    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240196    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240200    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240204    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240207    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240210    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240214    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240217    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240220    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240223    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240226    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240229    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240232    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240236    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240240    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240242    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240246    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240249    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240252    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240255    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 16:36:23.245664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240258    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240263    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240266    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240269    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240273    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240276    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240279    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240282    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240286    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240289    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240292    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240295    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240298    2577 flags.go:64] FLAG: --enable-server="true"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240301    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240305    2577 flags.go:64] FLAG: --event-burst="100"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240307    2577 flags.go:64] FLAG: --event-qps="50"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240310    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240313    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240316    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240320    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240323    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240325    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240329    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240332    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240334    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 16:36:23.246308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240339    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240342    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240344    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240347    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240350    2577 flags.go:64] FLAG: --feature-gates=""
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240354    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240356    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240359    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240363    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240366    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240369    2577 flags.go:64] FLAG: --help="false"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240373    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240376    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240392    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240397    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240402    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240407    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240411    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240414    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240417    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240420    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240423    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240426    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240428    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 16 16:36:23.246942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240431    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240434    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240437    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240440    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240443    2577 flags.go:64] FLAG: --lock-file=""
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240445    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240448    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240451    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240457    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240460    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240463    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240466    2577 flags.go:64] FLAG: --logging-format="text"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240469    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240472    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240475    2577 flags.go:64] FLAG: --manifest-url=""
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240478    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240482    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240485    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240490    2577 flags.go:64] FLAG: --max-pods="110"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240494    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240497    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240500    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240503    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240506    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240509    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 16:36:23.247549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240512    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240519    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240522    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240525    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240528    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240531    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240536    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240539    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240542    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240545    2577 flags.go:64] FLAG: --port="10250"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240548    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240551    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0453bc7a970a371d7"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240554    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240557    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240560    2577 flags.go:64] FLAG: --register-node="true"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240563    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240573    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240576    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240579    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240582    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240585    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240588    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240591    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240594    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240597    2577 flags.go:64] FLAG: --runonce="false"
Sep 16 16:36:23.248223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240600    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240604    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240607    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240610    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240613    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240615    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240618    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240621    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240624    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240627    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240630    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240632    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240636    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240639    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240641    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240647    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240650    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240653    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240657    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240659    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240662    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240665    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240668    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240670    2577 flags.go:64] FLAG: --v="2"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240674    2577 flags.go:64] FLAG: --version="false"
Sep 16 16:36:23.248891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240679    2577 flags.go:64] FLAG: --vmodule=""
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240682    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.240685    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240801    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240805    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240808    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240811    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240814    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240818    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240820    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240825    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240828    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240831    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240835    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240839    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240842    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240845    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240848    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240851    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.249521 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240854    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240856    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240859    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240862    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240864    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240867    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240869    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240872    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240875    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240877    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240880    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240882    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240885    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240888    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240891    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240893    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240896    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240898    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240901    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.249983 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240905    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240908    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240912    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240915    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240919    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240922    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240925    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240928    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240931    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240934    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240937    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240940    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240942    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240945    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240948    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240951    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240953    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240956    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240959    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.250466 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240961    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240964    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240966    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240969    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240972    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240975    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240977    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240980    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240982    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240985    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240988    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240990    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240993    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.240997    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241000    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241003    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241006    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241009    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241012    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241014    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.250931 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241017    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241019    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241022    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241024    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241027    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241029    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241032    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241035    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241037    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241040    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241043    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241045    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241048    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241050    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241053    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241055    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241058    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241061    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241063    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241066    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.251421 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241069    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241072    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241075    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241077    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.241080    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.241086    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.247843    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.247859    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.248699    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.248711    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248765    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248770    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248774    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248777    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.251923 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248780    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248784    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248789    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248792    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248795    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248798    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248801    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248803    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248806    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248809    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248812    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248814    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248817    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248819    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248822    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248826    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248829    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248832    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248836    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.252298 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248839    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248841    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248844    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248847    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248850    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248853    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248857    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248860    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248863    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248866    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248869    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248871    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248874    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248876    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248879    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248881    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248884    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248886    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248889    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248891    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.252846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248894    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248897    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248899    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248902    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248905    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248908    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248910    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248913    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248916    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248918    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248921    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248924    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248926    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248929    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248931    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248934    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248937    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248940    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248943    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.253333 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248946    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248949    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248952    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248954    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248957    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248959    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248962    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248964    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248967    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248970    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248972    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248975    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248977    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248980    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248983    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248986    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248988    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248991    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248993    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248996    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.253815 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.248999    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249001    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249004    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249007    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249009    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249012    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249015    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249019    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249022    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249025    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249029    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249031    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249034    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249037    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249040    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249042    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.254310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249045    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.249050    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249156    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249160    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249163    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249166    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249169    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249172    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249174    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249177    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249180    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249182    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249185    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249188    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249190    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:23.254713 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249193    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249195    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249198    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249201    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249203    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249206    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249208    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249211    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249213    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249216    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249219    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249221    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249224    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249226    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249229    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249232    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249235    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249237    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249240    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249242    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:23.255085 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249245    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249248    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249250    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249253    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249255    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249258    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249260    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249263    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249265    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249268    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249270    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249273    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249275    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249278    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249281    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249283    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249286    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249289    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249291    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249294    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:23.255586 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249296    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249299    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249303    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249307    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249310    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249313    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249316    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249318    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249322    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249324    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249327    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249329    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249332    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249334    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249337    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249339    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249342    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249344    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249346    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:23.256073 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249349    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249351    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249354    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249356    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249359    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249361    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249364    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249366    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249369    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249372    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249375    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249392    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249397    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249401    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249404    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249407    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249409    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249412    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249415    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:23.256588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249417    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249421    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249425    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249428    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249431    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249434    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249437    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:23.249440    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.249445    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.249551    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.251918    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.253105    2577 server.go:1019] "Starting client certificate rotation"
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.253211    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 16:36:23.257037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.253247    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 16:36:23.257398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.255893    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 16:36:23.259930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.259868    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 16:36:23.267550 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.267534    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 16 16:36:23.270573 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.270558    2577 log.go:25] "Validated CRI v1 image API"
Sep 16 16:36:23.274523 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.274507    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 16:36:23.276972 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.276953    2577 fs.go:135] Filesystem UUIDs: map[21b6105a-c828-4c72-a4a7-84e620855c22:/dev/nvme0n1p3 528cc241-faa5-4499-9828-5bee6371618b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 16 16:36:23.277043 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.276971    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 16:36:23.282008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.281908    2577 manager.go:216] Machine: {Timestamp:2026-09-16 16:36:23.280469903 +0000 UTC m=+0.366189967 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099070 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287a0feff8bff8ba3a0508f79d9de4 SystemUUID:ec287a0f-eff8-bff8-ba3a-0508f79d9de4 BootID:fea442ad-0fa2-4f8a-990b-8d74bc210fcd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c6:04:c7:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c6:04:c7:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:55:5c:c5:b9:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 16:36:23.282008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.282007    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 16:36:23.282109 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.282070    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 16:36:23.285494 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.285479    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 16:36:23.285543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.285491    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 16:36:23.285658 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.285521    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 16:36:23.286389 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.286370    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 16:36:23.286427 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.286394    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 16:36:23.286427 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.286401    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 16:36:23.287248 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.287237    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 16:36:23.288446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.288436    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 16:36:23.292754 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292741    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 16:36:23.292866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292854    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 16:36:23.292903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292886    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 16:36:23.292903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292897    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 16:36:23.292955 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292910    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 16:36:23.292955 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292918    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 16 16:36:23.292955 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.292927    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 16:36:23.293255 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.293241    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 16:36:23.293315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.293270    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 16:36:23.296161 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.296112    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 16:36:23.299178 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.299162    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 16:36:23.299220 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.299195    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 16:36:23.301270 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301258    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301274    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301280    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301286    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301294    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301300    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 16:36:23.301321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301318    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 16:36:23.301570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301327    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 16:36:23.301570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301344    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 16:36:23.301570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301351    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 16:36:23.301570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301359    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 16:36:23.301570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.301365    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 16:36:23.302518 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.302489    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 16:36:23.302518 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.302515    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 16:36:23.306332 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.306312    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 16:36:23.306491 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.306449    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 16:36:23.306592 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.306558    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 16:36:23.306787 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.306777    2577 server.go:1268] "Started kubelet"
Sep 16 16:36:23.306862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.306819    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 16:36:23.307031 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.306988    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 16:36:23.307102 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.307054    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 16:36:23.307821 ip-10-0-132-17 systemd[1]: Started Kubernetes Kubelet.
Sep 16 16:36:23.309422 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.309283    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 16:36:23.310936 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.310006    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 16:36:23.312676 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.312568    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 16:36:23.316808 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.315677    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-17.ec2.internal.18d5da6bc25ae60a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-17.ec2.internal,UID:ip-10-0-132-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-17.ec2.internal,},FirstTimestamp:2026-09-16 16:36:23.306733066 +0000 UTC m=+0.392453132,LastTimestamp:2026-09-16 16:36:23.306733066 +0000 UTC m=+0.392453132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-17.ec2.internal,}"
Sep 16 16:36:23.317863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.317845    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 16:36:23.318529 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.318514    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 16:36:23.318675 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.318645    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.319272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319250    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 16:36:23.319272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319251    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 16:36:23.319436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319285    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 16:36:23.319436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319262    2577 factory.go:55] Registering systemd factory
Sep 16 16:36:23.319436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319336    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 16 16:36:23.319436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319359    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 16:36:23.319436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319367    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 16:36:23.319670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319540    2577 factory.go:153] Registering CRI-O factory
Sep 16 16:36:23.319670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319555    2577 factory.go:223] Registration of the crio container factory successfully
Sep 16 16:36:23.319670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319598    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 16:36:23.319670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319612    2577 factory.go:103] Registering Raw factory
Sep 16 16:36:23.319670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319626    2577 manager.go:1196] Started watching for new ooms in manager
Sep 16 16:36:23.319989 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.319976    2577 manager.go:349] Starting recovery of all containers
Sep 16 16:36:23.330043 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.330009    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 16:36:23.330132 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.330075    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 16:36:23.330329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.330313    2577 manager.go:354] Recovery completed
Sep 16 16:36:23.335935 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.335821    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.337963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.337947    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.338037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.337980    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.338037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.337996    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.338647 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.338624    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 16:36:23.338647 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.338638    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.338765 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.338640    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 16:36:23.338765 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.338687    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 16:36:23.339261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.339248    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.339322 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.339275    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.339322 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.339284    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.340805 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.340789    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hgb2"
Sep 16 16:36:23.340873 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.340809    2577 policy_none.go:49] "None policy: Start"
Sep 16 16:36:23.340873 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.340822    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 16:36:23.340873 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.340831    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 16:36:23.342737 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.342722    2577 policy_none.go:47] "Start"
Sep 16 16:36:23.349329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.349305    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hgb2"
Sep 16 16:36:23.351531 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.351469    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-17.ec2.internal.18d5da6bc4376c4b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-17.ec2.internal,UID:ip-10-0-132-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-17.ec2.internal,},FirstTimestamp:2026-09-16 16:36:23.337962571 +0000 UTC m=+0.423682639,LastTimestamp:2026-09-16 16:36:23.337962571 +0000 UTC m=+0.423682639,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-17.ec2.internal,}"
Sep 16 16:36:23.375952 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.375936    2577 manager.go:337] "Starting Device Plugin manager"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.375976    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.375988    2577 server.go:87] "Starting device plugin registration server"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.376285    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.376297    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.376437    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.376511    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.376521    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.376990    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.377031    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.382957    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.384050    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.384104    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.384137    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.384206    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 16:36:23.394943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.386161    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:23.476929 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.476908    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.477708 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.477688    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.477708 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.477709    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.477824 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.477718    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.477824 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.477742    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.484988 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.484942    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"]
Sep 16 16:36:23.485047 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.484999    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.485720 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.485705    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.485786 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.485732    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.485786 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.485746    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.487336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.487322    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.487424 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.487346    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-17.ec2.internal\": node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.487761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.487750    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.487864 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.487841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.487898 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.487871    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.488370 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488356    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.488447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488371    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.488447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488397    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.488447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488407    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.488447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488421    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.488447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.488410    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.490349 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.490335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.490421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.490361    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:23.490980 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.490962    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:23.491041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.490989    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:23.491041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.490998    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:23.514573 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.514555    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.520726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.520708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.520784 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.520734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.520784 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.520755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cba30f3283a64fbc23b4dd01a869002-config\") pod \"kube-apiserver-proxy-ip-10-0-132-17.ec2.internal\" (UID: \"6cba30f3283a64fbc23b4dd01a869002\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.534010 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.533996    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-17.ec2.internal\" not found" node="ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.538259 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.538245    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-17.ec2.internal\" not found" node="ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.614774 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.614756    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.621106 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.621169 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.621169 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cba30f3283a64fbc23b4dd01a869002-config\") pod \"kube-apiserver-proxy-ip-10-0-132-17.ec2.internal\" (UID: \"6cba30f3283a64fbc23b4dd01a869002\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.621237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cba30f3283a64fbc23b4dd01a869002-config\") pod \"kube-apiserver-proxy-ip-10-0-132-17.ec2.internal\" (UID: \"6cba30f3283a64fbc23b4dd01a869002\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.621237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.621237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.621194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db9b3544cb38d600a3f64c8b5063a0c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal\" (UID: \"db9b3544cb38d600a3f64c8b5063a0c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.715261 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.715239    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.816018 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.815962    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:23.837497 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.837476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.842453 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:23.842435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:23.916844 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:23.916822    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.017395 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:24.017368    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.117896 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:24.117843    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.219786 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:24.219750    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.252438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.252403    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 16:36:24.252844 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.252553    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 16:36:24.320168 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:24.320137    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.355198 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.355151    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 16:31:23 +0000 UTC" deadline="2028-02-27 19:24:17.030797794 +0000 UTC"
Sep 16 16:36:24.355198 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.355191    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12698h47m52.675611342s"
Sep 16 16:36:24.385465 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.385398    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:24.420516 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:24.420498    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-17.ec2.internal\" not found"
Sep 16 16:36:24.439621 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.439599    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:24.519422 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.519378    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:24.530698 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.530675    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 16:36:24.532841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.532824    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal"
Sep 16 16:36:24.548000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.547973    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 16:36:24.563083 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:24.563053    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9b3544cb38d600a3f64c8b5063a0c8.slice/crio-b6941db48708ba432d3440247f128445fa33b2aa58d86f7e751fa9afe5f462c9 WatchSource:0}: Error finding container b6941db48708ba432d3440247f128445fa33b2aa58d86f7e751fa9afe5f462c9: Status 404 returned error can't find the container with id b6941db48708ba432d3440247f128445fa33b2aa58d86f7e751fa9afe5f462c9
Sep 16 16:36:24.570234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.570199    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:36:24.573235 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:24.573216    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cba30f3283a64fbc23b4dd01a869002.slice/crio-724c31daaacb738a3e3097377aea05aef308f5c7102165a0168d31301e321907 WatchSource:0}: Error finding container 724c31daaacb738a3e3097377aea05aef308f5c7102165a0168d31301e321907: Status 404 returned error can't find the container with id 724c31daaacb738a3e3097377aea05aef308f5c7102165a0168d31301e321907
Sep 16 16:36:24.853472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:24.853411    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:25.076998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.076970    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:25.294040 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.293979    2577 apiserver.go:52] "Watching apiserver"
Sep 16 16:36:25.298931 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.298909    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 16:36:25.299321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.299299    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c","openshift-dns/node-resolver-2vkc2","openshift-image-registry/node-ca-pkhf5","openshift-multus/multus-additional-cni-plugins-lxlc9","openshift-multus/multus-qzkn2","openshift-multus/network-metrics-daemon-z78gp","openshift-ovn-kubernetes/ovnkube-node-x42jz","kube-system/global-pull-secret-syncer-56zhm","kube-system/konnectivity-agent-t8fd8","openshift-cluster-node-tuning-operator/tuned-qsfk2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal","openshift-network-diagnostics/network-check-target-gvz5j","openshift-network-operator/iptables-alerter-2cchc","kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal"]
Sep 16 16:36:25.302008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.301987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.304491 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304456    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.304638 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 16:36:25.304638 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.304767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304465    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 16:36:25.304767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 16:36:25.304767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 16:36:25.304923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304576    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-srf27\""
Sep 16 16:36:25.304923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.304469    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.306879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.306856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.306987 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.306970    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.307290 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.307263    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-996wx\""
Sep 16 16:36:25.307382 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.307311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 16:36:25.307464 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.307381    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.309376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.309316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.309608 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.309589    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.310208 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.309873    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.310208 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.309915    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 16:36:25.310208 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.310180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c6bp9\""
Sep 16 16:36:25.311310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.311288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 16:36:25.311750 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.311732    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.314470 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.314452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.314568 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.314527    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:25.315536 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.315501    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 16:36:25.316424 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.316401    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.316534 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.316405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 16:36:25.317051 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.317032    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.317376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.317360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pbcsj\""
Sep 16 16:36:25.318438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.317563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m255f\""
Sep 16 16:36:25.318438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.317735    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 16:36:25.318438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.318087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.318438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.318127    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 16:36:25.320480 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.320458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hj6r7\""
Sep 16 16:36:25.321008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.320848    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 16:36:25.321008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.320884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 16:36:25.321008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.320959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.323830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.323762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.324304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.324282    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.324597 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.324537    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 16:36:25.324697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.324541    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2xcn8\""
Sep 16 16:36:25.324697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.324542    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.326322 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.326303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.327004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.326984    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m9d2z\""
Sep 16 16:36:25.328683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.328631    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.328683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.328676    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4mxg\""
Sep 16 16:36:25.328822 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.328690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.329153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.328975    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:36:25.329255 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 16:36:25.329324 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.329380 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.329324    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:25.329633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.329740 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.329740 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-systemd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.329740 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-ovn\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.329900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-script-lib\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.329900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-system-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.329900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhrxm\" (UniqueName: \"kubernetes.io/projected/8c86855e-120a-4581-9302-9b30b70d1364-kube-api-access-mhrxm\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.329900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-netd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5fdb\" (UniqueName: \"kubernetes.io/projected/83ab5800-2241-45f2-b350-c023353623eb-kube-api-access-v5fdb\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-conf-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.330061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.329990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a55152b-766f-402d-a86f-ad31b37d9e66-konnectivity-ca\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.330061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-multus\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-kubelet-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-bin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-socket-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.330228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgpdw\" (UniqueName: \"kubernetes.io/projected/8a7b0509-a48f-420d-964c-caeebd79c4b8-kube-api-access-sgpdw\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f746c9-f470-40d9-9f34-f495ada964a9-serviceca\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a55152b-766f-402d-a86f-ad31b37d9e66-agent-certs\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdl29\" (UniqueName: \"kubernetes.io/projected/ab13dd64-c766-4023-b466-5816400a9273-kube-api-access-rdl29\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-var-lib-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-node-log\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.330434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-multus-certs\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-kubelet\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-env-overrides\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2pv\" (UniqueName: \"kubernetes.io/projected/a0f746c9-f470-40d9-9f34-f495ada964a9-kube-api-access-8v2pv\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-system-cni-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-etc-kubernetes\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-systemd-units\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-slash\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-etc-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-cni-binary-copy\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-netns\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-kubelet\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ba71fb0-bd57-4680-a677-facb11f8d2df-iptables-alerter-script\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-sys-fs\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.331999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-hostroot\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.330957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4klxc\" (UniqueName: \"kubernetes.io/projected/8ba71fb0-bd57-4680-a677-facb11f8d2df-kube-api-access-4klxc\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-log-socket\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-bin\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7mr6\" (UniqueName: \"kubernetes.io/projected/fae52386-5999-4306-a8fd-5281677d700e-kube-api-access-v7mr6\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ba71fb0-bd57-4680-a677-facb11f8d2df-host-slash\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-device-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-netns\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-cnibin\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-multus-daemon-config\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-registration-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f746c9-f470-40d9-9f34-f495ada964a9-host\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-cnibin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-socket-dir-parent\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-config\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.332808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83ab5800-2241-45f2-b350-c023353623eb-ovn-node-metrics-cert\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.333557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-k8s-cni-cncf-io\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.333557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-etc-selinux\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.333557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.331931    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:25.333557 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.331985    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:25.333557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.333007    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 16:36:25.364530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.364506    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vs2wv"
Sep 16 16:36:25.376245 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.375899    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vs2wv"
Sep 16 16:36:25.389959 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.389913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal" event={"ID":"db9b3544cb38d600a3f64c8b5063a0c8","Type":"ContainerStarted","Data":"b6941db48708ba432d3440247f128445fa33b2aa58d86f7e751fa9afe5f462c9"}
Sep 16 16:36:25.391421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.391384    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal" event={"ID":"6cba30f3283a64fbc23b4dd01a869002","Type":"ContainerStarted","Data":"724c31daaacb738a3e3097377aea05aef308f5c7102165a0168d31301e321907"}
Sep 16 16:36:25.420976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.420950    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 16:36:25.432165 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-env-overrides\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2pv\" (UniqueName: \"kubernetes.io/projected/a0f746c9-f470-40d9-9f34-f495ada964a9-kube-api-access-8v2pv\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.432272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-system-cni-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.432272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-etc-kubernetes\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sjhr\" (UniqueName: \"kubernetes.io/projected/e60f338f-568d-49d7-bec5-3e15b5ff8f53-kube-api-access-9sjhr\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-systemd-units\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-system-cni-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-slash\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-etc-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-etc-kubernetes\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-systemd-units\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-cni-binary-copy\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-slash\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-netns\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-etc-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-kubelet\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ba71fb0-bd57-4680-a677-facb11f8d2df-iptables-alerter-script\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-sys-fs\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-kubelet\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-hostroot\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4klxc\" (UniqueName: \"kubernetes.io/projected/8ba71fb0-bd57-4680-a677-facb11f8d2df-kube-api-access-4klxc\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-netns\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.432726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysconfig\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-run\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-lib-modules\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-tmp\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-log-socket\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-bin\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432833    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-env-overrides\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7mr6\" (UniqueName: \"kubernetes.io/projected/fae52386-5999-4306-a8fd-5281677d700e-kube-api-access-v7mr6\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-sys-fs\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ba71fb0-bd57-4680-a677-facb11f8d2df-host-slash\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-kubelet-config\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-dbus\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.432941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-hostroot\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-log-socket\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-bin\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433140    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-cni-binary-copy\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-device-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.433484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ba71fb0-bd57-4680-a677-facb11f8d2df-host-slash\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-netns\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ba71fb0-bd57-4680-a677-facb11f8d2df-iptables-alerter-script\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-cnibin\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-cnibin\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-netns\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-multus-daemon-config\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-device-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-modprobe-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-var-lib-kubelet\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-registration-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f746c9-f470-40d9-9f34-f495ada964a9-host\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-cnibin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-socket-dir-parent\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-registration-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.434339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e60f338f-568d-49d7-bec5-3e15b5ff8f53-tmp-dir\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-kubernetes\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0f746c9-f470-40d9-9f34-f495ada964a9-host\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-cnibin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-config\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c86855e-120a-4581-9302-9b30b70d1364-multus-daemon-config\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83ab5800-2241-45f2-b350-c023353623eb-ovn-node-metrics-cert\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-k8s-cni-cncf-io\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-socket-dir-parent\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e60f338f-568d-49d7-bec5-3e15b5ff8f53-hosts-file\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-k8s-cni-cncf-io\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-etc-selinux\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.433976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-systemd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-etc-selinux\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.435194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-ovn\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-script-lib\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-ovn\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-system-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-run-systemd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.434256    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fae52386-5999-4306-a8fd-5281677d700e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.434316    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:25.934285331 +0000 UTC m=+3.020005386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhrxm\" (UniqueName: \"kubernetes.io/projected/8c86855e-120a-4581-9302-9b30b70d1364-kube-api-access-mhrxm\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-systemd\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434348    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-host\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-netd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434385    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-system-cni-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-cni-netd\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5fdb\" (UniqueName: \"kubernetes.io/projected/83ab5800-2241-45f2-b350-c023353623eb-kube-api-access-v5fdb\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436067 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434491    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-conf-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a55152b-766f-402d-a86f-ad31b37d9e66-konnectivity-ca\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-multus\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-sys\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-script-lib\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-kubelet-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-bin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-bin\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-var-lib-cni-multus\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-conf\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw5gb\" (UniqueName: \"kubernetes.io/projected/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-kube-api-access-fw5gb\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-kubelet-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-multus-conf-dir\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.436991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-socket-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-tuned\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgpdw\" (UniqueName: \"kubernetes.io/projected/8a7b0509-a48f-420d-964c-caeebd79c4b8-kube-api-access-sgpdw\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f746c9-f470-40d9-9f34-f495ada964a9-serviceca\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a55152b-766f-402d-a86f-ad31b37d9e66-agent-certs\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab13dd64-c766-4023-b466-5816400a9273-socket-dir\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdl29\" (UniqueName: \"kubernetes.io/projected/ab13dd64-c766-4023-b466-5816400a9273-kube-api-access-rdl29\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-var-lib-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-node-log\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-multus-certs\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-kubelet\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a55152b-766f-402d-a86f-ad31b37d9e66-konnectivity-ca\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-host-kubelet\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-var-lib-openvswitch\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83ab5800-2241-45f2-b350-c023353623eb-node-log\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.437859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c86855e-120a-4581-9302-9b30b70d1364-host-run-multus-certs\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.438757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0f746c9-f470-40d9-9f34-f495ada964a9-serviceca\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.438757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.438757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.434354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83ab5800-2241-45f2-b350-c023353623eb-ovnkube-config\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.438757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.435785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fae52386-5999-4306-a8fd-5281677d700e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.438757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.438676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83ab5800-2241-45f2-b350-c023353623eb-ovn-node-metrics-cert\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.439042 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.438837    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a55152b-766f-402d-a86f-ad31b37d9e66-agent-certs\") pod \"konnectivity-agent-t8fd8\" (UID: \"6a55152b-766f-402d-a86f-ad31b37d9e66\") " pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.444739 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.444709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2pv\" (UniqueName: \"kubernetes.io/projected/a0f746c9-f470-40d9-9f34-f495ada964a9-kube-api-access-8v2pv\") pod \"node-ca-pkhf5\" (UID: \"a0f746c9-f470-40d9-9f34-f495ada964a9\") " pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.445200 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.445177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4klxc\" (UniqueName: \"kubernetes.io/projected/8ba71fb0-bd57-4680-a677-facb11f8d2df-kube-api-access-4klxc\") pod \"iptables-alerter-2cchc\" (UID: \"8ba71fb0-bd57-4680-a677-facb11f8d2df\") " pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.446292 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.446271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7mr6\" (UniqueName: \"kubernetes.io/projected/fae52386-5999-4306-a8fd-5281677d700e-kube-api-access-v7mr6\") pod \"multus-additional-cni-plugins-lxlc9\" (UID: \"fae52386-5999-4306-a8fd-5281677d700e\") " pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.450030 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.450010    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhrxm\" (UniqueName: \"kubernetes.io/projected/8c86855e-120a-4581-9302-9b30b70d1364-kube-api-access-mhrxm\") pod \"multus-qzkn2\" (UID: \"8c86855e-120a-4581-9302-9b30b70d1364\") " pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.450849 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.450587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5fdb\" (UniqueName: \"kubernetes.io/projected/83ab5800-2241-45f2-b350-c023353623eb-kube-api-access-v5fdb\") pod \"ovnkube-node-x42jz\" (UID: \"83ab5800-2241-45f2-b350-c023353623eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.451739 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.451714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgpdw\" (UniqueName: \"kubernetes.io/projected/8a7b0509-a48f-420d-964c-caeebd79c4b8-kube-api-access-sgpdw\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.453640 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.453608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdl29\" (UniqueName: \"kubernetes.io/projected/ab13dd64-c766-4023-b466-5816400a9273-kube-api-access-rdl29\") pod \"aws-ebs-csi-driver-node-78h9c\" (UID: \"ab13dd64-c766-4023-b466-5816400a9273\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.535579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-kubelet-config\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.535710 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-dbus\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.535710 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-modprobe-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.535710 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-kubelet-config\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-modprobe-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-dbus\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-var-lib-kubelet\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e60f338f-568d-49d7-bec5-3e15b5ff8f53-tmp-dir\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-kubernetes\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.535879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-d\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-var-lib-kubelet\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-kubernetes\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e60f338f-568d-49d7-bec5-3e15b5ff8f53-hosts-file\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-systemd\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.535997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-host\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e60f338f-568d-49d7-bec5-3e15b5ff8f53-hosts-file\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-sys\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-conf\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-systemd\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw5gb\" (UniqueName: \"kubernetes.io/projected/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-kube-api-access-fw5gb\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-host\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-sys\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e60f338f-568d-49d7-bec5-3e15b5ff8f53-tmp-dir\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-tuned\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sjhr\" (UniqueName: \"kubernetes.io/projected/e60f338f-568d-49d7-bec5-3e15b5ff8f53-kube-api-access-9sjhr\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.536210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysctl-conf\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysconfig\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-run\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-lib-modules\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-sysconfig\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-tmp\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-run\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.536479    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.536538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-lib-modules\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.537107 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.536546    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:26.036526337 +0000 UTC m=+3.122246412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:25.539431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.539394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-etc-tuned\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.539575 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.539559    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-tmp\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.544417 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.544360    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:25.544417 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.544401    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:25.544417 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.544416    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:25.544611 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.544465    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:26.044451897 +0000 UTC m=+3.130171949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:25.548502 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.548481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw5gb\" (UniqueName: \"kubernetes.io/projected/2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0-kube-api-access-fw5gb\") pod \"tuned-qsfk2\" (UID: \"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0\") " pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.548502 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.548490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sjhr\" (UniqueName: \"kubernetes.io/projected/e60f338f-568d-49d7-bec5-3e15b5ff8f53-kube-api-access-9sjhr\") pod \"node-resolver-2vkc2\" (UID: \"e60f338f-568d-49d7-bec5-3e15b5ff8f53\") " pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.614090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.614061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:25.623447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.623426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2cchc"
Sep 16 16:36:25.625658 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.625634    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ab5800_2241_45f2_b350_c023353623eb.slice/crio-8a5226322908737b1095c2731d7e4ed50fc22fcee3eccc9ee217da188cc6e112 WatchSource:0}: Error finding container 8a5226322908737b1095c2731d7e4ed50fc22fcee3eccc9ee217da188cc6e112: Status 404 returned error can't find the container with id 8a5226322908737b1095c2731d7e4ed50fc22fcee3eccc9ee217da188cc6e112
Sep 16 16:36:25.631738 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.631714    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba71fb0_bd57_4680_a677_facb11f8d2df.slice/crio-a146500883eefaf9c72f789a5e01cf55d8fc811a29ed57be2705cd3b075f1e33 WatchSource:0}: Error finding container a146500883eefaf9c72f789a5e01cf55d8fc811a29ed57be2705cd3b075f1e33: Status 404 returned error can't find the container with id a146500883eefaf9c72f789a5e01cf55d8fc811a29ed57be2705cd3b075f1e33
Sep 16 16:36:25.634964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.634945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pkhf5"
Sep 16 16:36:25.640336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.640316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lxlc9"
Sep 16 16:36:25.644703 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.644683    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f746c9_f470_40d9_9f34_f495ada964a9.slice/crio-07f9470ad8309f2265c2d15084855828d4f3ebcf27a6ebfeebed0811f27e4f7a WatchSource:0}: Error finding container 07f9470ad8309f2265c2d15084855828d4f3ebcf27a6ebfeebed0811f27e4f7a: Status 404 returned error can't find the container with id 07f9470ad8309f2265c2d15084855828d4f3ebcf27a6ebfeebed0811f27e4f7a
Sep 16 16:36:25.649761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.649732    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qzkn2"
Sep 16 16:36:25.649978 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.649959    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae52386_5999_4306_a8fd_5281677d700e.slice/crio-ec817f700630367e6a17abc719223d742be67e2c08ed1b1c9b124ae416cf4693 WatchSource:0}: Error finding container ec817f700630367e6a17abc719223d742be67e2c08ed1b1c9b124ae416cf4693: Status 404 returned error can't find the container with id ec817f700630367e6a17abc719223d742be67e2c08ed1b1c9b124ae416cf4693
Sep 16 16:36:25.658114 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.658092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:25.666315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.666293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c"
Sep 16 16:36:25.667178 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.667158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a55152b_766f_402d_a86f_ad31b37d9e66.slice/crio-723f6d37dd8217f756146ed95244f5715416ae06cffe92e90daab00677ef4da0 WatchSource:0}: Error finding container 723f6d37dd8217f756146ed95244f5715416ae06cffe92e90daab00677ef4da0: Status 404 returned error can't find the container with id 723f6d37dd8217f756146ed95244f5715416ae06cffe92e90daab00677ef4da0
Sep 16 16:36:25.674497 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.674473    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab13dd64_c766_4023_b466_5816400a9273.slice/crio-9ef37a8cae1fb28618a0b5af391a033b66f8154c14f4af5fbac6f3b1b70ed119 WatchSource:0}: Error finding container 9ef37a8cae1fb28618a0b5af391a033b66f8154c14f4af5fbac6f3b1b70ed119: Status 404 returned error can't find the container with id 9ef37a8cae1fb28618a0b5af391a033b66f8154c14f4af5fbac6f3b1b70ed119
Sep 16 16:36:25.675243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.675215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2vkc2"
Sep 16 16:36:25.682191 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.682173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qsfk2"
Sep 16 16:36:25.683796 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.683773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60f338f_568d_49d7_bec5_3e15b5ff8f53.slice/crio-a47a5ccda04ab0a26924569cfecd1d0c7bf557c9735639277ebabf8573e7798a WatchSource:0}: Error finding container a47a5ccda04ab0a26924569cfecd1d0c7bf557c9735639277ebabf8573e7798a: Status 404 returned error can't find the container with id a47a5ccda04ab0a26924569cfecd1d0c7bf557c9735639277ebabf8573e7798a
Sep 16 16:36:25.691656 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:25.691469    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d57cd14_5f5c_4e75_b0a0_7b50dbdc2db0.slice/crio-484845c2c80afdde61a2a381f425d3b7a0bca4268c107176473762c06416098a WatchSource:0}: Error finding container 484845c2c80afdde61a2a381f425d3b7a0bca4268c107176473762c06416098a: Status 404 returned error can't find the container with id 484845c2c80afdde61a2a381f425d3b7a0bca4268c107176473762c06416098a
Sep 16 16:36:25.938291 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:25.938216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:25.938457 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.938402    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:25.938524 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:25.938482    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:26.938461853 +0000 UTC m=+4.024181919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:26.038997 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.038960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:26.039129 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.039107    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:26.039185 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.039178    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:27.039159309 +0000 UTC m=+4.124879360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:26.139270 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.139244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:26.139460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.139369    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:26.139460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.139397    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:26.139460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.139406    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:26.139460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.139457    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:27.139440012 +0000 UTC m=+4.225160079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:26.379746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.379640    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 16:31:25 +0000 UTC" deadline="2028-02-21 16:14:52.398082226 +0000 UTC"
Sep 16 16:36:26.379746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.379698    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12551h38m26.018387949s"
Sep 16 16:36:26.394349 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.394315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2vkc2" event={"ID":"e60f338f-568d-49d7-bec5-3e15b5ff8f53","Type":"ContainerStarted","Data":"a47a5ccda04ab0a26924569cfecd1d0c7bf557c9735639277ebabf8573e7798a"}
Sep 16 16:36:26.395675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.395640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c" event={"ID":"ab13dd64-c766-4023-b466-5816400a9273","Type":"ContainerStarted","Data":"9ef37a8cae1fb28618a0b5af391a033b66f8154c14f4af5fbac6f3b1b70ed119"}
Sep 16 16:36:26.396984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.396959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t8fd8" event={"ID":"6a55152b-766f-402d-a86f-ad31b37d9e66","Type":"ContainerStarted","Data":"723f6d37dd8217f756146ed95244f5715416ae06cffe92e90daab00677ef4da0"}
Sep 16 16:36:26.398267 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.398242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qzkn2" event={"ID":"8c86855e-120a-4581-9302-9b30b70d1364","Type":"ContainerStarted","Data":"085f47698730632fd348c87ca107eca344ae5f5b64b29323280ada1682966a1a"}
Sep 16 16:36:26.399535 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.399508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerStarted","Data":"ec817f700630367e6a17abc719223d742be67e2c08ed1b1c9b124ae416cf4693"}
Sep 16 16:36:26.400824 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.400799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pkhf5" event={"ID":"a0f746c9-f470-40d9-9f34-f495ada964a9","Type":"ContainerStarted","Data":"07f9470ad8309f2265c2d15084855828d4f3ebcf27a6ebfeebed0811f27e4f7a"}
Sep 16 16:36:26.402065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.402042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qsfk2" event={"ID":"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0","Type":"ContainerStarted","Data":"484845c2c80afdde61a2a381f425d3b7a0bca4268c107176473762c06416098a"}
Sep 16 16:36:26.403246 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.403225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2cchc" event={"ID":"8ba71fb0-bd57-4680-a677-facb11f8d2df","Type":"ContainerStarted","Data":"a146500883eefaf9c72f789a5e01cf55d8fc811a29ed57be2705cd3b075f1e33"}
Sep 16 16:36:26.404574 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.404542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"8a5226322908737b1095c2731d7e4ed50fc22fcee3eccc9ee217da188cc6e112"}
Sep 16 16:36:26.946525 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:26.945879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:26.946525 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.946071    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:26.946525 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:26.946134    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:28.946115487 +0000 UTC m=+6.031835560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:27.046630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.046599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:27.046768 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.046744    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.046834 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.046801    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:29.046783583 +0000 UTC m=+6.132503646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.147317 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.147243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:27.147472 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.147438    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:27.147472 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.147461    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:27.147472 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.147473    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.147648 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.147537    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:29.147517926 +0000 UTC m=+6.233237990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.381444 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.380085    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 16:31:25 +0000 UTC" deadline="2028-06-11 07:05:19.097706854 +0000 UTC"
Sep 16 16:36:27.381444 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.380123    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15206h28m51.717587973s"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.385429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.385554    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.385931    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.386019    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.386080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:27.387426 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:27.386146    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:27.423497 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.423249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal" event={"ID":"6cba30f3283a64fbc23b4dd01a869002","Type":"ContainerStarted","Data":"97bd7490c6b8304d4beebd96573166c9a0f7cc4c6a03a805e87e74b8131cc0c8"}
Sep 16 16:36:27.460813 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:27.460762    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-17.ec2.internal" podStartSLOduration=3.460742164 podStartE2EDuration="3.460742164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:27.460599461 +0000 UTC m=+4.546319535" watchObservedRunningTime="2026-09-16 16:36:27.460742164 +0000 UTC m=+4.546462269"
Sep 16 16:36:28.434662 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:28.434624    2577 generic.go:356] "Generic (PLEG): container finished" podID="db9b3544cb38d600a3f64c8b5063a0c8" containerID="c9ae7f06be67064321c79be3ee9ddce07b5032b962b2c1f2384eb8530cdc3aad" exitCode=0
Sep 16 16:36:28.435231 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:28.434745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal" event={"ID":"db9b3544cb38d600a3f64c8b5063a0c8","Type":"ContainerDied","Data":"c9ae7f06be67064321c79be3ee9ddce07b5032b962b2c1f2384eb8530cdc3aad"}
Sep 16 16:36:28.962131 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:28.961269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:28.962131 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:28.961432    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:28.962131 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:28.961495    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:32.961476704 +0000 UTC m=+10.047196760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:29.063596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:29.062944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:29.063596 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.063164    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:29.063596 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.063231    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:33.063211494 +0000 UTC m=+10.148931545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:29.164609 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:29.163985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:29.164609 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.164181    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:29.164609 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.164199    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:29.164609 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.164211    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:29.164609 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.164265    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:33.164247481 +0000 UTC m=+10.249967550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:29.385708 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:29.385625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:29.385855 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:29.385625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:29.385855 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.385846    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:29.385955 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.385759    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:29.385955 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:29.385910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:29.386070 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:29.385970    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:31.385371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:31.385427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:31.385394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:31.385519    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:31.385599    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:31.385762 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:31.385699    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:32.998537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:32.998482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:32.999106 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:32.998635    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:32.999106 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:32.998703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:40.998683069 +0000 UTC m=+18.084403128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:33.099583 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:33.099553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:33.099713 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.099682    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:33.099765 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.099735    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:41.099715442 +0000 UTC m=+18.185435498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:33.200239 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:33.200198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:33.200457 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.200437    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:33.200532 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.200466    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:33.200532 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.200480    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:33.200639 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.200542    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:41.20052451 +0000 UTC m=+18.286244574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:33.389269 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:33.389197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:33.389618 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.389591    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:33.390250 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:33.390227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:33.390611 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.390590    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:33.390800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:33.390786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:33.391016 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:33.390978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:35.385148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:35.384994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:35.385148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:35.384994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:35.385148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:35.385129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:35.385148 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:35.385126    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:35.385858 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:35.385257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:35.385858 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:35.385341    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:37.385220 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:37.385187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:37.385669 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:37.385193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:37.385669 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:37.385296    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:37.385669 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:37.385193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:37.385669 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:37.385377    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:37.385669 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:37.385445    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:39.384619 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:39.384589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:39.385055 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:39.384701    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:39.385055 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:39.384588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:39.385055 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:39.384747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:39.385055 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:39.384842    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:39.385055 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:39.384925    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:41.062924 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.062895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:41.063398 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.062994    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:41.063398 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.063043    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.063029124 +0000 UTC m=+34.148749179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:41.163751 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.163725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:41.163912 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.163853    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:41.163971 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.163915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.163896414 +0000 UTC m=+34.249616478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:41.264547 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.264512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:41.264673 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.264638    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:41.264673 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.264662    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:41.264673 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.264673    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:41.264820 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.264724    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.26471191 +0000 UTC m=+34.350431962 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:41.384916 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.384837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:41.385062 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.384965    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:41.385062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.385010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:41.385175 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.385107    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:41.385175 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:41.385140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:41.385270 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:41.385235    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:43.385310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:43.385275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:43.385780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:43.385404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:43.385780 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:43.385409    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:43.385780 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:43.385508    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:43.385780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:43.385543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:43.385780 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:43.385620    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:44.469138 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.468824    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal" event={"ID":"db9b3544cb38d600a3f64c8b5063a0c8","Type":"ContainerStarted","Data":"96f8e4c716e02f4b30c9b840eef5edc23f4997459ee9b5305faf1c3364f4f320"}
Sep 16 16:36:44.470281 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.470261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2vkc2" event={"ID":"e60f338f-568d-49d7-bec5-3e15b5ff8f53","Type":"ContainerStarted","Data":"8ef2d66cc491e00cb953f27fc685daaa5c4290be9cdfdcce10382d8ea4d60991"}
Sep 16 16:36:44.471520 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.471499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c" event={"ID":"ab13dd64-c766-4023-b466-5816400a9273","Type":"ContainerStarted","Data":"c66cc9c62fae5d418c227b272a6c62d26480e977900eb203f7e5590342a00d42"}
Sep 16 16:36:44.472851 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.472827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t8fd8" event={"ID":"6a55152b-766f-402d-a86f-ad31b37d9e66","Type":"ContainerStarted","Data":"7c9d5d4295225a324b5afddc5486a0322b0ebb3b3291b143f0e1acf3d5762409"}
Sep 16 16:36:44.474184 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.474165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qzkn2" event={"ID":"8c86855e-120a-4581-9302-9b30b70d1364","Type":"ContainerStarted","Data":"87a04c45811b9a5a4e9039bd0ee03948bf7fb042531554ec508a155019f1166d"}
Sep 16 16:36:44.475619 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.475600    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="bd4db66225a9f2632c9cb318cafe6ecff77a760e46fd62f047c3a3be2b1595cd" exitCode=0
Sep 16 16:36:44.475693 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.475660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"bd4db66225a9f2632c9cb318cafe6ecff77a760e46fd62f047c3a3be2b1595cd"}
Sep 16 16:36:44.477270 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.477246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pkhf5" event={"ID":"a0f746c9-f470-40d9-9f34-f495ada964a9","Type":"ContainerStarted","Data":"d400a7a040ac226e420ea037f844111d8f37f4bfc00a561e66e436519cb0896b"}
Sep 16 16:36:44.478797 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.478769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qsfk2" event={"ID":"2d57cd14-5f5c-4e75-b0a0-7b50dbdc2db0","Type":"ContainerStarted","Data":"c10420e73fe729bcd7c16c3d77840c6bc6fd224d408c1a22cdc8f6a0bd9f94b3"}
Sep 16 16:36:44.481079 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.481061    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"9d4e4518cee47412c9b240a00c1e1355eace337849f89936f561074a49829dda"}
Sep 16 16:36:44.481183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.481113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"9d58637441b47f86fef9d4c16e120cb58c3c5f89becc652f700e670968906005"}
Sep 16 16:36:44.481183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.481141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"685eb31354c6f79a86ab2b851193a6e73a0e7f329126af5dea70b2342148006b"}
Sep 16 16:36:44.481183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.481168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"868f273099cb35f2650f701ab5a103b55ad43e4f0af68818df79bb7d71b22387"}
Sep 16 16:36:44.488180 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.488145    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-17.ec2.internal" podStartSLOduration=20.488133645 podStartE2EDuration="20.488133645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.487766348 +0000 UTC m=+21.573486421" watchObservedRunningTime="2026-09-16 16:36:44.488133645 +0000 UTC m=+21.573853717"
Sep 16 16:36:44.536031 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.535994    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qzkn2" podStartSLOduration=3.2283023379999998 podStartE2EDuration="21.535981479s" totalImagesPullingTime="18.307679141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.511309737 +0000 UTC m=+21.597029809" watchObservedRunningTime="2026-09-16 16:36:44.535981479 +0000 UTC m=+21.621701573"
Sep 16 16:36:44.560235 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.560182    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t8fd8" podStartSLOduration=7.473073428 podStartE2EDuration="21.560162158s" totalImagesPullingTime="14.08708873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.535715926 +0000 UTC m=+21.621436000" watchObservedRunningTime="2026-09-16 16:36:44.560162158 +0000 UTC m=+21.645882233"
Sep 16 16:36:44.560321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.560292    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qsfk2" podStartSLOduration=3.326943521 podStartE2EDuration="21.560283341s" totalImagesPullingTime="18.23333982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.559659386 +0000 UTC m=+21.645379458" watchObservedRunningTime="2026-09-16 16:36:44.560283341 +0000 UTC m=+21.646003415"
Sep 16 16:36:44.578970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.578937    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pkhf5" podStartSLOduration=3.312222562 podStartE2EDuration="21.578925858s" totalImagesPullingTime="18.266703296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.578804392 +0000 UTC m=+21.664524464" watchObservedRunningTime="2026-09-16 16:36:44.578925858 +0000 UTC m=+21.664645959"
Sep 16 16:36:44.658182 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:44.657087    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2vkc2" podStartSLOduration=3.433943617 podStartE2EDuration="21.657065287s" totalImagesPullingTime="18.22312167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:44.597257448 +0000 UTC m=+21.682977521" watchObservedRunningTime="2026-09-16 16:36:44.657065287 +0000 UTC m=+21.742785361"
Sep 16 16:36:45.384671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.384645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:45.384671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.384667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:45.384823 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.384645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:45.384823 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:45.384775    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:45.384925 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:45.384846    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:45.384976 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:45.384944    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:45.453150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.453129    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 16:36:45.485407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.485337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2cchc" event={"ID":"8ba71fb0-bd57-4680-a677-facb11f8d2df","Type":"ContainerStarted","Data":"6936082e16d5853debf0731b80b3ddc03ced1d984307fb1686eb1d13827c8cf1"}
Sep 16 16:36:45.488820 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.488788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"cd2ac294452370742c6a695d052864834a62a3d160143eb4d835ce8644017947"}
Sep 16 16:36:45.488925 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.488856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"87e28d8dd4caf540b392e89d14d68581406c2bd093e215001f3c07804ee5267a"}
Sep 16 16:36:45.490914 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:45.490881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c" event={"ID":"ab13dd64-c766-4023-b466-5816400a9273","Type":"ContainerStarted","Data":"bee8e2113f4a28f990854a5c8169fcf9ea74726e97f0263f919ca1e159efe31c"}
Sep 16 16:36:46.387714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:46.387616    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T16:36:45.453146544Z","UUID":"c63dbaaf-7858-4269-8355-5fc3852e761b","Handler":null,"Name":"","Endpoint":""}
Sep 16 16:36:46.389282 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:46.389261    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 16:36:46.389444 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:46.389291    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 16:36:47.387918 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.387745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:47.388440 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.387745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:47.388440 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:47.388010    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:47.388440 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.387749    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:47.388440 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:47.388056    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:47.388440 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:47.388160    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:47.499157 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.499125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c" event={"ID":"ab13dd64-c766-4023-b466-5816400a9273","Type":"ContainerStarted","Data":"4779e85a349037e95b2cad7c0fc7d6f7fd26a7ef976e6d86ea61dc75036422cd"}
Sep 16 16:36:47.502454 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.502433    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"d81f7f0a6135b714375c769fdaad2531915937aba7329d8a4a17ed11e64b1e37"}
Sep 16 16:36:47.522435 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.522396    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2cchc" podStartSLOduration=6.231331147 podStartE2EDuration="24.52234161s" totalImagesPullingTime="18.291010463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:45.50377314 +0000 UTC m=+22.589493217" watchObservedRunningTime="2026-09-16 16:36:47.52234161 +0000 UTC m=+24.608061666"
Sep 16 16:36:47.948192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.948168    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:47.948658 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.948638    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:47.966319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:47.966277    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-78h9c" podStartSLOduration=4.6848223 podStartE2EDuration="24.966263992s" totalImagesPullingTime="20.281441692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.522887876 +0000 UTC m=+24.608607949" watchObservedRunningTime="2026-09-16 16:36:47.966263992 +0000 UTC m=+25.051984064"
Sep 16 16:36:48.504877 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:48.504851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:48.505450 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:48.505426    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t8fd8"
Sep 16 16:36:49.384865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:49.384829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:49.385038 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:49.384844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:49.385038 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:49.384962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:49.385038 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:49.384966    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:49.385202 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:49.385039    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:49.385202 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:49.385112    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:50.513746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.513682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" event={"ID":"83ab5800-2241-45f2-b350-c023353623eb","Type":"ContainerStarted","Data":"a49ec3551e06eb882cb3c6c574681b6743da4d8b08c89ea9b2800162838790e4"}
Sep 16 16:36:50.514126 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.513899    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:50.515478 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.515453    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="f39ecb3a32d15a1e14ebee0141d2c224578c87e4179cd0c9b2a9e5f202ae2ad1" exitCode=0
Sep 16 16:36:50.515575 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.515544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"f39ecb3a32d15a1e14ebee0141d2c224578c87e4179cd0c9b2a9e5f202ae2ad1"}
Sep 16 16:36:50.533643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.533624    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:50.545164 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:50.545114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz" podStartSLOduration=9.046388166 podStartE2EDuration="27.545100608s" totalImagesPullingTime="18.498712442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:50.54470601 +0000 UTC m=+27.630426084" watchObservedRunningTime="2026-09-16 16:36:50.545100608 +0000 UTC m=+27.630820679"
Sep 16 16:36:51.384491 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.384460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:51.384621 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.384469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:51.384621 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:51.384558    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:51.384737 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.384468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:51.384737 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:51.384632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:51.384829 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:51.384763    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:51.520651 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.520602    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:51.521284 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.520854    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:51.537907 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:51.537880    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:36:52.197880 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.197822    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gvz5j"]
Sep 16 16:36:52.197973 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.197909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:52.198082 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:52.197994    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:52.201943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.201916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-56zhm"]
Sep 16 16:36:52.202056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.202003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:52.202126 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:52.202087    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:52.202506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.202485    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z78gp"]
Sep 16 16:36:52.202596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.202557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:52.202673 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:52.202646    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:52.523627 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.523561    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="3499898a655f7f238f22a7819bce891c4be5cd97a748ef4892d2a8b7b09eade4" exitCode=0
Sep 16 16:36:52.524239 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:52.523646    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"3499898a655f7f238f22a7819bce891c4be5cd97a748ef4892d2a8b7b09eade4"}
Sep 16 16:36:54.385565 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:54.385334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:54.385861 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:54.385406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:54.385861 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:54.385648    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:54.385861 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:54.385443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:54.385861 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:54.385711    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:54.385861 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:54.385804    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:54.530553 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:54.530511    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="35c1b53841ad50044a80c70ce4a4fa9012aa8ef205acb31e11a32ab34f5967d7" exitCode=0
Sep 16 16:36:54.530651 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:54.530582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"35c1b53841ad50044a80c70ce4a4fa9012aa8ef205acb31e11a32ab34f5967d7"}
Sep 16 16:36:56.384542 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.384513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:56.384542 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.384529    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:56.385104 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.384610    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:36:56.385104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.384692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:56.385104 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.384785    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gvz5j" podUID="e50343cf-7d6e-421e-983b-ed80196eab72"
Sep 16 16:36:56.385104 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.384877    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56zhm" podUID="dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad"
Sep 16 16:36:56.727195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.727130    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-17.ec2.internal" event="NodeReady"
Sep 16 16:36:56.727316 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.727226    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 16:36:56.773001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.772974    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:36:56.789930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.789907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.792343 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.792322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:36:56.794421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.793464    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"]
Sep 16 16:36:56.794421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.793504    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 16:36:56.794421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.793901    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sm9dz\""
Sep 16 16:36:56.794421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.794146    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 16:36:56.795418 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.794882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 16:36:56.800089 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.800071    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 16:36:56.809981 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.809962    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b55tq"]
Sep 16 16:36:56.810093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.810074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:56.812207 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.812156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 16:36:56.812207 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.812179    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v2gqr\""
Sep 16 16:36:56.812374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.812323    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 16:36:56.832720 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.832697    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w65qs"]
Sep 16 16:36:56.833340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.832812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:56.835441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.835200    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 16:36:56.835441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.835234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqrg9\""
Sep 16 16:36:56.835648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.835630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 16:36:56.854836 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.854818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"]
Sep 16 16:36:56.854936 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.854842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b55tq"]
Sep 16 16:36:56.854936 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.854858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w65qs"]
Sep 16 16:36:56.855044 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.854935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:56.858072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.858053    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 16:36:56.858165 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.858101    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 16:36:56.858344 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.858329    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 16:36:56.858528 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.858504    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6pkw2\""
Sep 16 16:36:56.886747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886849 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886849 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886948 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886948 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886948 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.886948 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.887170 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.886999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jncsv\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988220 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988220 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988220 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jncsv\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf712-f78e-4aca-84be-6a24fae7c992-tmp-dir\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt55d\" (UniqueName: \"kubernetes.io/projected/87ecf712-f78e-4aca-84be-6a24fae7c992-kube-api-access-vt55d\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xptdp\" (UniqueName: \"kubernetes.io/projected/c6d1931b-2d51-4f56-97bb-45e75ab928af-kube-api-access-xptdp\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:56.988463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/87ecf712-f78e-4aca-84be-6a24fae7c992-config-volume\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6d1931b-2d51-4f56-97bb-45e75ab928af-cert\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.988800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.989107 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.988832    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:36:56.989107 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.988847    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:36:56.989107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.988834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.989107 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:56.988908    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.488890747 +0000 UTC m=+34.574610811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:36:56.989340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.989323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.992213 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.992198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.992277 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.992231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.998472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.998442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:56.998650 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:56.998635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncsv\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:57.089636 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.089755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/87ecf712-f78e-4aca-84be-6a24fae7c992-config-volume\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.089755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6d1931b-2d51-4f56-97bb-45e75ab928af-cert\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:57.089755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:57.089914 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.089760    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:36:57.089914 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf712-f78e-4aca-84be-6a24fae7c992-tmp-dir\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.089914 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt55d\" (UniqueName: \"kubernetes.io/projected/87ecf712-f78e-4aca-84be-6a24fae7c992-kube-api-access-vt55d\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.089914 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:57.090104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:57.090104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.089937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xptdp\" (UniqueName: \"kubernetes.io/projected/c6d1931b-2d51-4f56-97bb-45e75ab928af-kube-api-access-xptdp\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:57.090104 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.090015    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.589995275 +0000 UTC m=+34.675715330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:36:57.090104 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.090077    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:57.090299 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.090112    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:36:57.090299 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.090127    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:29.090113723 +0000 UTC m=+66.175833787 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:57.090299 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.090168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:36:57.590152159 +0000 UTC m=+34.675872224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:36:57.090299 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.090186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf712-f78e-4aca-84be-6a24fae7c992-tmp-dir\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.090299 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.090290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/87ecf712-f78e-4aca-84be-6a24fae7c992-config-volume\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.090713 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.090691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:57.092142 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.092121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6d1931b-2d51-4f56-97bb-45e75ab928af-cert\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:57.109478 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.109460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xptdp\" (UniqueName: \"kubernetes.io/projected/c6d1931b-2d51-4f56-97bb-45e75ab928af-kube-api-access-xptdp\") pod \"ingress-canary-w65qs\" (UID: \"c6d1931b-2d51-4f56-97bb-45e75ab928af\") " pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:57.114357 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.114337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt55d\" (UniqueName: \"kubernetes.io/projected/87ecf712-f78e-4aca-84be-6a24fae7c992-kube-api-access-vt55d\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.164238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.164218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w65qs"
Sep 16 16:36:57.190223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.190203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:57.190314 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.190294    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:57.190371 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.190334    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret podName:dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad nodeName:}" failed. No retries permitted until 2026-09-16 16:37:29.190322506 +0000 UTC m=+66.276042557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret") pod "global-pull-secret-syncer-56zhm" (UID: "dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:57.290942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.290907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:57.291074 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.291039    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:57.291074 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.291057    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:57.291074 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.291066    2577 projected.go:196] Error preparing data for projected volume kube-api-access-p9mks for pod openshift-network-diagnostics/network-check-target-gvz5j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:57.291225 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.291116    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks podName:e50343cf-7d6e-421e-983b-ed80196eab72 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:29.291097622 +0000 UTC m=+66.376817673 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9mks" (UniqueName: "kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks") pod "network-check-target-gvz5j" (UID: "e50343cf-7d6e-421e-983b-ed80196eab72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:57.478513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.478489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w65qs"]
Sep 16 16:36:57.482141 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:36:57.482116    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d1931b_2d51_4f56_97bb_45e75ab928af.slice/crio-960a780cb45445331f4fde7eb706f46b66be6b4805e387d9a72b2f6edc019a57 WatchSource:0}: Error finding container 960a780cb45445331f4fde7eb706f46b66be6b4805e387d9a72b2f6edc019a57: Status 404 returned error can't find the container with id 960a780cb45445331f4fde7eb706f46b66be6b4805e387d9a72b2f6edc019a57
Sep 16 16:36:57.491971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.491951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:57.492102 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.492086    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:36:57.492138 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.492106    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:36:57.492171 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.492150    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.492136045 +0000 UTC m=+35.577856095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:36:57.538258 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.538223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w65qs" event={"ID":"c6d1931b-2d51-4f56-97bb-45e75ab928af","Type":"ContainerStarted","Data":"960a780cb45445331f4fde7eb706f46b66be6b4805e387d9a72b2f6edc019a57"}
Sep 16 16:36:57.593047 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.593022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:57.593149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:57.593055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:57.593206 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.593173    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:36:57.593206 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.593177    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:36:57.593301 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.593234    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.593216267 +0000 UTC m=+35.678936322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:36:57.593301 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:57.593260    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.593251397 +0000 UTC m=+35.678971450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:36:58.385033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.385001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:36:58.385346 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.385002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:36:58.385346 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.385001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:36:58.387982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.387772    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 16:36:58.387982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.387813    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 16:36:58.387982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.387867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 16:36:58.388643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.388623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6lts\""
Sep 16 16:36:58.388766 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.388628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtfpq\""
Sep 16 16:36:58.388766 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.388725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 16:36:58.500062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.500033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:36:58.500586 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.500188    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:36:58.500586 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.500202    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:36:58.500586 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.500263    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:37:00.500241257 +0000 UTC m=+37.585961320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:36:58.601126 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.601095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:36:58.601268 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:36:58.601143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:36:58.601268 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.601210    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:36:58.601376 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.601293    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:37:00.601263109 +0000 UTC m=+37.686983174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:36:58.601460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.601397    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:36:58.601460 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:36:58.601454    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:00.601430707 +0000 UTC m=+37.687150772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:00.515844 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:00.515805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:37:00.516291 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.515970    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:00.516291 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.515990    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:37:00.516291 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.516054    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:37:04.516038444 +0000 UTC m=+41.601758493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:37:00.617112 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:00.617075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:37:00.617270 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:00.617119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:37:00.617270 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.617225    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:37:00.617270 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.617245    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:00.617450 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.617288    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:37:04.617273583 +0000 UTC m=+41.702993633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:37:00.617450 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:00.617304    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:04.617296947 +0000 UTC m=+41.703016997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:01.551333 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:01.551299    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="15e73369d871444dd66699b18384b645a7e30d66463a52b54371739d350381d1" exitCode=0
Sep 16 16:37:01.551928 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:01.551394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"15e73369d871444dd66699b18384b645a7e30d66463a52b54371739d350381d1"}
Sep 16 16:37:01.553071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:01.553048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w65qs" event={"ID":"c6d1931b-2d51-4f56-97bb-45e75ab928af","Type":"ContainerStarted","Data":"a4b659465ace840107ae575c089f90c6f0109fd93c3a32e9530eea22f8458954"}
Sep 16 16:37:01.591969 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:01.591921    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w65qs" podStartSLOduration=2.116534557 podStartE2EDuration="5.591903783s" totalImagesPullingTime="3.475369226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:01.591457088 +0000 UTC m=+38.677177161" watchObservedRunningTime="2026-09-16 16:37:01.591903783 +0000 UTC m=+38.677623858"
Sep 16 16:37:02.558930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:02.558742    2577 generic.go:356] "Generic (PLEG): container finished" podID="fae52386-5999-4306-a8fd-5281677d700e" containerID="3766cf5204054b35103eb6d04e0e4df6ab31acd9862f18f685a74deb2a924f90" exitCode=0
Sep 16 16:37:02.559307 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:02.558828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerDied","Data":"3766cf5204054b35103eb6d04e0e4df6ab31acd9862f18f685a74deb2a924f90"}
Sep 16 16:37:03.564169 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:03.564135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" event={"ID":"fae52386-5999-4306-a8fd-5281677d700e","Type":"ContainerStarted","Data":"8cd7b5f7c46b574ce53497fe24701fbe66a11a73d5a3f9291cb6f334582b0d7f"}
Sep 16 16:37:03.594990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:03.594935    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lxlc9" podStartSLOduration=8.078107697 podStartE2EDuration="40.594916256s" totalImagesPullingTime="32.516808559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:03.593539397 +0000 UTC m=+40.679259468" watchObservedRunningTime="2026-09-16 16:37:03.594916256 +0000 UTC m=+40.680636329"
Sep 16 16:37:04.547592 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:04.547544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:37:04.547776 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.547678    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:04.547776 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.547697    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:37:04.547776 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.547743    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:37:12.547728277 +0000 UTC m=+49.633448327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:37:04.648697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:04.648672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:37:04.649026 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:04.648702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:37:04.649026 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.648802    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:37:04.649026 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.648828    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:04.649026 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.648863    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:37:12.648841332 +0000 UTC m=+49.734561382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:37:04.649026 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:04.648878    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:12.648871502 +0000 UTC m=+49.734591552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:12.603772 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:12.603734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:37:12.604148 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.603873    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:12.604148 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.603890    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:37:12.604148 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.603945    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:37:28.603928701 +0000 UTC m=+65.689648756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:37:12.704562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:12.704529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:37:12.704562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:12.704562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:37:12.704765 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.704695    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:12.704765 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.704703    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:37:12.704765 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.704742    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:28.70472899 +0000 UTC m=+65.790449040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:12.704881 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:12.704771    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:37:28.70475202 +0000 UTC m=+65.790472107 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:37:23.544335 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:23.544307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x42jz"
Sep 16 16:37:28.618459 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:28.618418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:37:28.618977 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.618539    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:28.618977 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.618551    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:37:28.618977 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.618617    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:38:00.618600188 +0000 UTC m=+97.704320237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:37:28.718872 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:28.718834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:37:28.718872 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:28.718876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:37:28.718998 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.718982    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:37:28.719041 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.719018    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:28.719080 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.719051    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:38:00.719033081 +0000 UTC m=+97.804753209 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:37:28.719080 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:28.719068    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:38:00.719061217 +0000 UTC m=+97.804781267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:29.121029 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.120998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:37:29.123552 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.123529    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 16:37:29.131444 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:29.131420    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:37:29.131611 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:37:29.131502    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:38:33.131481547 +0000 UTC m=+130.217201597 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : secret "metrics-daemon-secret" not found
Sep 16 16:37:29.221595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.221558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:37:29.224002 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.223982    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 16:37:29.235354 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.235330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad-original-pull-secret\") pod \"global-pull-secret-syncer-56zhm\" (UID: \"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad\") " pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:37:29.312995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.312969    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56zhm"
Sep 16 16:37:29.322289 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.322266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:37:29.324845 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.324821    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 16:37:29.335703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.335683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 16:37:29.346188 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.346164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mks\" (UniqueName: \"kubernetes.io/projected/e50343cf-7d6e-421e-983b-ed80196eab72-kube-api-access-p9mks\") pod \"network-check-target-gvz5j\" (UID: \"e50343cf-7d6e-421e-983b-ed80196eab72\") " pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:37:29.585593 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.585567    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-56zhm"]
Sep 16 16:37:29.589473 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:37:29.589442    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf2eb2e_913a_4d2a_9b7b_f82bc24971ad.slice/crio-33003e106184c059a16845c6b00a206b1f8a60a739c5e0d1f9dbe32569134899 WatchSource:0}: Error finding container 33003e106184c059a16845c6b00a206b1f8a60a739c5e0d1f9dbe32569134899: Status 404 returned error can't find the container with id 33003e106184c059a16845c6b00a206b1f8a60a739c5e0d1f9dbe32569134899
Sep 16 16:37:29.608839 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.608817    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtfpq\""
Sep 16 16:37:29.616734 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.616712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:37:29.619273 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.619251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-56zhm" event={"ID":"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad","Type":"ContainerStarted","Data":"33003e106184c059a16845c6b00a206b1f8a60a739c5e0d1f9dbe32569134899"}
Sep 16 16:37:29.883679 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:29.883618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gvz5j"]
Sep 16 16:37:29.886215 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:37:29.886195    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50343cf_7d6e_421e_983b_ed80196eab72.slice/crio-3c6b3bfb8d77aa6b04d118abd03430c9e6e13bb6b1393b1a011c832de807f008 WatchSource:0}: Error finding container 3c6b3bfb8d77aa6b04d118abd03430c9e6e13bb6b1393b1a011c832de807f008: Status 404 returned error can't find the container with id 3c6b3bfb8d77aa6b04d118abd03430c9e6e13bb6b1393b1a011c832de807f008
Sep 16 16:37:30.623663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:30.623628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gvz5j" event={"ID":"e50343cf-7d6e-421e-983b-ed80196eab72","Type":"ContainerStarted","Data":"3c6b3bfb8d77aa6b04d118abd03430c9e6e13bb6b1393b1a011c832de807f008"}
Sep 16 16:37:34.635570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:34.635538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gvz5j" event={"ID":"e50343cf-7d6e-421e-983b-ed80196eab72","Type":"ContainerStarted","Data":"b61c3b246c7cc3271563e5c94a67f844eb1fc4615844cdc2fe712ba4bf2ee25d"}
Sep 16 16:37:34.635921 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:34.635683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:37:35.640597 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:35.640552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-56zhm" event={"ID":"dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad","Type":"ContainerStarted","Data":"a251673ab806125bf51d2f86af6a6ad9b2c63eae0ba8200552f9fcb9551ac4a7"}
Sep 16 16:37:35.660446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:35.660379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gvz5j" podStartSLOduration=67.996259067 podStartE2EDuration="1m12.660364438s" totalImagesPullingTime="4.664105371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:34.653919496 +0000 UTC m=+71.739639568" watchObservedRunningTime="2026-09-16 16:37:35.660364438 +0000 UTC m=+72.746084596"
Sep 16 16:37:35.660822 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:37:35.660793    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-56zhm" podStartSLOduration=67.692830086 podStartE2EDuration="1m12.660785081s" totalImagesPullingTime="4.967954995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:35.659308851 +0000 UTC m=+72.745028923" watchObservedRunningTime="2026-09-16 16:37:35.660785081 +0000 UTC m=+72.746505226"
Sep 16 16:38:00.642395 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:00.642354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:38:00.642823 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.642487    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:38:00.642823 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.642501    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:38:00.642823 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.642567    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:39:04.642550768 +0000 UTC m=+161.728270818 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:38:00.743527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:00.743495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:38:00.743527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:00.743528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:38:00.743705 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.743615    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:38:00.743705 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.743632    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:38:00.743705 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.743673    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:39:04.743660378 +0000 UTC m=+161.829380433 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:38:00.743705 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:00.743688    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:39:04.743681134 +0000 UTC m=+161.829401184 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:38:05.643466 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:05.643404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gvz5j"
Sep 16 16:38:14.673531 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.673492    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"]
Sep 16 16:38:14.675484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.675460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.677741 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.677718    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 16:38:14.677876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.677747    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 16:38:14.678835 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.678811    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 16:38:14.678952 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.678864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 16:38:14.689966 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.689944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"]
Sep 16 16:38:14.837970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.837947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfv8f\" (UniqueName: \"kubernetes.io/projected/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-kube-api-access-sfv8f\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.838091 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.837976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.838091 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.838014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-tmp\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.938589 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.938524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-tmp\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.938589 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.938592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfv8f\" (UniqueName: \"kubernetes.io/projected/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-kube-api-access-sfv8f\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.938755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.938612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.939018 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.938994    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-tmp\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.941166 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.941143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.947570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.947547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfv8f\" (UniqueName: \"kubernetes.io/projected/98ec44c5-ee7e-4f0d-8737-3926ddfbab6d-kube-api-access-sfv8f\") pod \"klusterlet-addon-workmgr-59c7c94488-mmd9s\" (UID: \"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:14.985483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:14.985455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:15.255357 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:15.255283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"]
Sep 16 16:38:15.258600 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:38:15.258576    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ec44c5_ee7e_4f0d_8737_3926ddfbab6d.slice/crio-6d66b120fdbdc644d861df5fbf2b5f00df745201c89d8fe0d342af06ef727033 WatchSource:0}: Error finding container 6d66b120fdbdc644d861df5fbf2b5f00df745201c89d8fe0d342af06ef727033: Status 404 returned error can't find the container with id 6d66b120fdbdc644d861df5fbf2b5f00df745201c89d8fe0d342af06ef727033
Sep 16 16:38:15.737770 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:15.737734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" event={"ID":"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d","Type":"ContainerStarted","Data":"6d66b120fdbdc644d861df5fbf2b5f00df745201c89d8fe0d342af06ef727033"}
Sep 16 16:38:19.750229 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:19.750193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" event={"ID":"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d","Type":"ContainerStarted","Data":"3c8449a63241526dfa6dcded8860c779de7f085801443463d7cd6685f2206aa7"}
Sep 16 16:38:19.750740 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:19.750507    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:19.752101 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:19.752074    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:38:19.772095 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:19.768654    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" podStartSLOduration=1.8649749469999999 podStartE2EDuration="5.768632804s" totalImagesPullingTime="3.903657857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:38:19.767056654 +0000 UTC m=+116.852776725" watchObservedRunningTime="2026-09-16 16:38:19.768632804 +0000 UTC m=+116.854352878"
Sep 16 16:38:33.157139 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:33.157095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:38:33.157607 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:33.157221    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:38:33.157607 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:33.157290    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:35.157272866 +0000 UTC m=+252.242992916 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : secret "metrics-daemon-secret" not found
Sep 16 16:38:59.810703 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:59.810655    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b"
Sep 16 16:38:59.822926 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:59.822902    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" podUID="86791c8b-0f46-4f45-a713-7f8dfe9d78fa"
Sep 16 16:38:59.842106 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:59.842076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:38:59.842106 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:38:59.842092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:38:59.842279 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:38:59.842160    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b55tq" podUID="87ecf712-f78e-4aca-84be-6a24fae7c992"
Sep 16 16:39:00.844062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:39:00.844022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:39:01.398830 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:01.398790    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:39:04.665080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:39:04.665044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:39:04.665519 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.665198    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:39:04.665519 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.665217    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf96db99c-l69ng: secret "image-registry-tls" not found
Sep 16 16:39:04.665519 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.665288    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls podName:c1287478-81eb-4b7c-a73f-89a7d26f296b nodeName:}" failed. No retries permitted until 2026-09-16 16:41:06.665269816 +0000 UTC m=+283.750989888 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls") pod "image-registry-6cf96db99c-l69ng" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b") : secret "image-registry-tls" not found
Sep 16 16:39:04.765570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:39:04.765532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:39:04.765570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:39:04.765570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:39:04.765738 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.765681    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:39:04.765738 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.765734    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:06.765721128 +0000 UTC m=+283.851441178 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:39:04.765822 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.765681    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:39:04.765865 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:39:04.765828    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:41:06.765814484 +0000 UTC m=+283.851534548 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:39:12.384422 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:39:12.384348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:40:19.751595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:19.751544    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" podUID="98ec44c5-ee7e-4f0d-8737-3926ddfbab6d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused"
Sep 16 16:40:20.027446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:20.027357    2577 generic.go:356] "Generic (PLEG): container finished" podID="98ec44c5-ee7e-4f0d-8737-3926ddfbab6d" containerID="3c8449a63241526dfa6dcded8860c779de7f085801443463d7cd6685f2206aa7" exitCode=1
Sep 16 16:40:20.027446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:20.027431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" event={"ID":"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d","Type":"ContainerDied","Data":"3c8449a63241526dfa6dcded8860c779de7f085801443463d7cd6685f2206aa7"}
Sep 16 16:40:20.027765 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:20.027748    2577 scope.go:117] "RemoveContainer" containerID="3c8449a63241526dfa6dcded8860c779de7f085801443463d7cd6685f2206aa7"
Sep 16 16:40:21.031990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:21.031956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s" event={"ID":"98ec44c5-ee7e-4f0d-8737-3926ddfbab6d","Type":"ContainerStarted","Data":"91277810d4cc5a2cc5f9c262f51c8c63cc4137bac9e23ae635ce69079a4dbd85"}
Sep 16 16:40:21.032462 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:21.032238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:40:21.032923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:21.032905    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59c7c94488-mmd9s"
Sep 16 16:40:35.182577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:35.182535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:40:35.182981 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:35.182672    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:40:35.182981 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:35.182741    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs podName:8a7b0509-a48f-420d-964c-caeebd79c4b8 nodeName:}" failed. No retries permitted until 2026-09-16 16:42:37.182726178 +0000 UTC m=+374.268446227 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs") pod "network-metrics-daemon-z78gp" (UID: "8a7b0509-a48f-420d-964c-caeebd79c4b8") : secret "metrics-daemon-secret" not found
Sep 16 16:40:52.440506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.440477    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-25qmb"]
Sep 16 16:40:52.443234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.443217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.447831 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.447801    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 16:40:52.451634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.451610    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.452787 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.452761    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.452882 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.452788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rdpz9\""
Sep 16 16:40:52.452882 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.452819    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 16:40:52.457139 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.457120    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 16:40:52.470716 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.469994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-25qmb"]
Sep 16 16:40:52.493901 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.493872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-tmp\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.494000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.493906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-trusted-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.494000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.493929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms99x\" (UniqueName: \"kubernetes.io/projected/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-kube-api-access-ms99x\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.494110 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.494013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-service-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.494110 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.494049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-snapshots\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.494110 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.494073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-serving-cert\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595191 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-tmp\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-trusted-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms99x\" (UniqueName: \"kubernetes.io/projected/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-kube-api-access-ms99x\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-service-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-snapshots\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-serving-cert\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595656 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-tmp\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-service-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.595957 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.595933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-snapshots\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.596082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.596067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-trusted-ca-bundle\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.597555 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.597537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-serving-cert\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.604134 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.604111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms99x\" (UniqueName: \"kubernetes.io/projected/8fd0d9db-5946-4d2e-9da3-3d0fe8246520-kube-api-access-ms99x\") pod \"insights-operator-5844b856db-25qmb\" (UID: \"8fd0d9db-5946-4d2e-9da3-3d0fe8246520\") " pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:52.751406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:52.751336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-25qmb"
Sep 16 16:40:53.018568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:53.018503    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-25qmb"]
Sep 16 16:40:53.022205 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:40:53.022175    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd0d9db_5946_4d2e_9da3_3d0fe8246520.slice/crio-61d0a502e37d6d60036c0e60116ff057c8ac0d2551d5fa223922a8c866d755f7 WatchSource:0}: Error finding container 61d0a502e37d6d60036c0e60116ff057c8ac0d2551d5fa223922a8c866d755f7: Status 404 returned error can't find the container with id 61d0a502e37d6d60036c0e60116ff057c8ac0d2551d5fa223922a8c866d755f7
Sep 16 16:40:53.105989 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:53.105964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-25qmb" event={"ID":"8fd0d9db-5946-4d2e-9da3-3d0fe8246520","Type":"ContainerStarted","Data":"61d0a502e37d6d60036c0e60116ff057c8ac0d2551d5fa223922a8c866d755f7"}
Sep 16 16:40:56.024557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.024525    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xp2rq"]
Sep 16 16:40:56.027471 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.027444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.029971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.029951    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 16:40:56.030081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.029954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nnmt\""
Sep 16 16:40:56.030157 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.030144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 16:40:56.039355 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.039335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp2rq"]
Sep 16 16:40:56.114041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.114008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-25qmb" event={"ID":"8fd0d9db-5946-4d2e-9da3-3d0fe8246520","Type":"ContainerStarted","Data":"6ae0ea90cb6358f9d45807708f1d1da21ea1603e5df4960f8580328656fedd65"}
Sep 16 16:40:56.120112 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.120081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b7b12f3-3483-4b5e-a990-efdbb2e82402-crio-socket\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.120195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.120129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwmq\" (UniqueName: \"kubernetes.io/projected/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-api-access-ccwmq\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.120195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.120157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.120271 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.120243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b7b12f3-3483-4b5e-a990-efdbb2e82402-data-volume\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.120271 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.120260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.129440 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.129403    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-25qmb" podStartSLOduration=2.080244373 podStartE2EDuration="4.129372627s" totalImagesPullingTime="2.049128254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:56.128975541 +0000 UTC m=+273.214695611" watchObservedRunningTime="2026-09-16 16:40:56.129372627 +0000 UTC m=+273.215092699"
Sep 16 16:40:56.220745 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.220718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b7b12f3-3483-4b5e-a990-efdbb2e82402-crio-socket\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.220868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.220772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwmq\" (UniqueName: \"kubernetes.io/projected/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-api-access-ccwmq\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.220868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.220806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.220947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.220870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b7b12f3-3483-4b5e-a990-efdbb2e82402-crio-socket\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.221003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.220988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b7b12f3-3483-4b5e-a990-efdbb2e82402-data-volume\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.221039 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.221004    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.221039 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.221012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.221132 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.221081    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.721062116 +0000 UTC m=+273.806782190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.221132 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.221105    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.221236 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.221157    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.721139807 +0000 UTC m=+273.806859876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.221480 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.221464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b7b12f3-3483-4b5e-a990-efdbb2e82402-data-volume\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.230729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.230708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwmq\" (UniqueName: \"kubernetes.io/projected/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-api-access-ccwmq\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.724154 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.724119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.724303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:56.724172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:56.724303 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.724271    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.724378 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.724319    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:57.724307422 +0000 UTC m=+274.810027478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.724378 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.724271    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.724471 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:56.724410    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:57.72439748 +0000 UTC m=+274.810117538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:57.448071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.448043    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"]
Sep 16 16:40:57.450971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.450955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"
Sep 16 16:40:57.453960 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.453937    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mzmjg\""
Sep 16 16:40:57.454083 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.453937    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:57.454083 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.454055    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 16 16:40:57.458704 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.458681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"]
Sep 16 16:40:57.530474 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.530448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm896\" (UniqueName: \"kubernetes.io/projected/4c6d3b0d-bd58-478a-a68c-cdc004050160-kube-api-access-vm896\") pod \"migrator-8b4d697c5-vqffb\" (UID: \"4c6d3b0d-bd58-478a-a68c-cdc004050160\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"
Sep 16 16:40:57.631651 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.631627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm896\" (UniqueName: \"kubernetes.io/projected/4c6d3b0d-bd58-478a-a68c-cdc004050160-kube-api-access-vm896\") pod \"migrator-8b4d697c5-vqffb\" (UID: \"4c6d3b0d-bd58-478a-a68c-cdc004050160\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"
Sep 16 16:40:57.640258 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.640232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm896\" (UniqueName: \"kubernetes.io/projected/4c6d3b0d-bd58-478a-a68c-cdc004050160-kube-api-access-vm896\") pod \"migrator-8b4d697c5-vqffb\" (UID: \"4c6d3b0d-bd58-478a-a68c-cdc004050160\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"
Sep 16 16:40:57.732569 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.732515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:57.732569 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.732561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:57.732683 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:57.732655    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:57.732718 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:57.732714    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:59.732701651 +0000 UTC m=+276.818421702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:57.732760 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:57.732656    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:57.732824 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:57.732812    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:59.732796584 +0000 UTC m=+276.818516639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:57.760489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.760465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"
Sep 16 16:40:57.778546 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:57.778527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2vkc2_e60f338f-568d-49d7-bec5-3e15b5ff8f53/dns-node-resolver/0.log"
Sep 16 16:40:58.029407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:58.029322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb"]
Sep 16 16:40:58.031993 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:40:58.031962    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6d3b0d_bd58_478a_a68c_cdc004050160.slice/crio-3ae3f3e09d2dfa48e3fe097d8ad489171fec8e30aa9823998f74ae783416437d WatchSource:0}: Error finding container 3ae3f3e09d2dfa48e3fe097d8ad489171fec8e30aa9823998f74ae783416437d: Status 404 returned error can't find the container with id 3ae3f3e09d2dfa48e3fe097d8ad489171fec8e30aa9823998f74ae783416437d
Sep 16 16:40:58.122007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:58.121976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb" event={"ID":"4c6d3b0d-bd58-478a-a68c-cdc004050160","Type":"ContainerStarted","Data":"3ae3f3e09d2dfa48e3fe097d8ad489171fec8e30aa9823998f74ae783416437d"}
Sep 16 16:40:58.979020 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:58.978996    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pkhf5_a0f746c9-f470-40d9-9f34-f495ada964a9/node-ca/0.log"
Sep 16 16:40:59.508016 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.507987    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-smrr4"]
Sep 16 16:40:59.510930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.510910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.514723 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.514704    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 16:40:59.514796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.514744    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 16:40:59.514963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.514950    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 16:40:59.515097 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.515081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 16:40:59.515482 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.515467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-657mn\""
Sep 16 16:40:59.529929 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.529876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-smrr4"]
Sep 16 16:40:59.545928 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.545909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2njp\" (UniqueName: \"kubernetes.io/projected/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-kube-api-access-p2njp\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.546017 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.545937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-cabundle\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.546062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.546013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-key\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.646908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.646881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2njp\" (UniqueName: \"kubernetes.io/projected/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-kube-api-access-p2njp\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.646908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.646907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-cabundle\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.647039 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.646976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-key\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.647474 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.647454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-cabundle\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.649185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.649169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-signing-key\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.655535 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.655515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2njp\" (UniqueName: \"kubernetes.io/projected/4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166-kube-api-access-p2njp\") pod \"service-ca-568869fdc5-smrr4\" (UID: \"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166\") " pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.747714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.747684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:59.747832 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.747748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:40:59.747832 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:59.747813    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:59.747928 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:59.747869    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:03.747854359 +0000 UTC m=+280.833574409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:59.747928 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:59.747876    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:59.747928 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:40:59.747929    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:03.747916559 +0000 UTC m=+280.833636694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:59.819829 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.819804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-smrr4"
Sep 16 16:40:59.978967 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:40:59.978942    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w65qs_c6d1931b-2d51-4f56-97bb-45e75ab928af/serve-healthcheck-canary/0.log"
Sep 16 16:41:00.095222 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:00.095163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-smrr4"]
Sep 16 16:41:00.098599 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:00.098562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9dcd25_1b20_44cc_bdc6_c1b5d08e1166.slice/crio-011e297ea1ea0bee40cffb97a4b56928f3059b78454284a187bc2df44595c376 WatchSource:0}: Error finding container 011e297ea1ea0bee40cffb97a4b56928f3059b78454284a187bc2df44595c376: Status 404 returned error can't find the container with id 011e297ea1ea0bee40cffb97a4b56928f3059b78454284a187bc2df44595c376
Sep 16 16:41:00.130490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:00.130462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb" event={"ID":"4c6d3b0d-bd58-478a-a68c-cdc004050160","Type":"ContainerStarted","Data":"6fb4f6ef31423dbd674062c4c0bf4ce713b0da12aac9aa29ad49c851cb337318"}
Sep 16 16:41:00.130601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:00.130513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb" event={"ID":"4c6d3b0d-bd58-478a-a68c-cdc004050160","Type":"ContainerStarted","Data":"a2c7e2f0562b216edba738e5014ae87cc8ed6636f7f08447ba25b0d3af264f58"}
Sep 16 16:41:00.131675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:00.131651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-smrr4" event={"ID":"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166","Type":"ContainerStarted","Data":"011e297ea1ea0bee40cffb97a4b56928f3059b78454284a187bc2df44595c376"}
Sep 16 16:41:00.150124 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:00.150082    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vqffb" podStartSLOduration=1.91300355 podStartE2EDuration="3.150069429s" totalImagesPullingTime="1.237065879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:00.149090513 +0000 UTC m=+277.234810585" watchObservedRunningTime="2026-09-16 16:41:00.150069429 +0000 UTC m=+277.235789539"
Sep 16 16:41:02.138831 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:02.138803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-smrr4" event={"ID":"4a9dcd25-1b20-44cc-bdc6-c1b5d08e1166","Type":"ContainerStarted","Data":"bb1ffdd8e7018048d490e28932761a011d5cc4d150bf76cc09f8144769d3bc14"}
Sep 16 16:41:02.159063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:02.158911    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-smrr4" podStartSLOduration=1.326297678 podStartE2EDuration="3.15889219s" totalImagesPullingTime="1.832594512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:02.15877039 +0000 UTC m=+279.244490462" watchObservedRunningTime="2026-09-16 16:41:02.15889219 +0000 UTC m=+279.244612253"
Sep 16 16:41:02.843156 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:02.843128    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" podUID="86791c8b-0f46-4f45-a713-7f8dfe9d78fa"
Sep 16 16:41:02.843260 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:02.843141    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b"
Sep 16 16:41:03.141895 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:03.141851    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:41:03.141895 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:03.141852    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:03.783974 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:03.783928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:03.784160 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:03.784070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:03.784251 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:03.784234    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:03.784344 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:03.784333    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:11.784314241 +0000 UTC m=+288.870034296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:03.784490 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:03.784470    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:41:03.784617 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:03.784607    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:11.784589857 +0000 UTC m=+288.870309922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : configmap "kube-rbac-proxy" not found
Sep 16 16:41:03.845352 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:03.845320    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b55tq" podUID="87ecf712-f78e-4aca-84be-6a24fae7c992"
Sep 16 16:41:04.145484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:04.145457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:41:06.703781 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.703744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:06.706437 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.706399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"image-registry-6cf96db99c-l69ng\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") " pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:06.745358 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.745330    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sm9dz\""
Sep 16 16:41:06.754045 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.754028    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:06.804912 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.804881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:41:06.805008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:06.804927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:41:06.805083 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:06.805067    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:41:06.805151 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:06.805138    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls podName:87ecf712-f78e-4aca-84be-6a24fae7c992 nodeName:}" failed. No retries permitted until 2026-09-16 16:43:08.805117173 +0000 UTC m=+405.890837287 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls") pod "dns-default-b55tq" (UID: "87ecf712-f78e-4aca-84be-6a24fae7c992") : secret "dns-default-metrics-tls" not found
Sep 16 16:41:06.805650 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:06.805630    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 16:41:06.805757 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:06.805705    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert podName:86791c8b-0f46-4f45-a713-7f8dfe9d78fa nodeName:}" failed. No retries permitted until 2026-09-16 16:43:08.805682812 +0000 UTC m=+405.891402869 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-smsn2" (UID: "86791c8b-0f46-4f45-a713-7f8dfe9d78fa") : secret "networking-console-plugin-cert" not found
Sep 16 16:41:07.025961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:07.025895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:41:07.029014 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:07.028990    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1287478_81eb_4b7c_a73f_89a7d26f296b.slice/crio-9959f6bba4511696db174ee4102adbbc43b9c822aeaa20a11ed414860977ff12 WatchSource:0}: Error finding container 9959f6bba4511696db174ee4102adbbc43b9c822aeaa20a11ed414860977ff12: Status 404 returned error can't find the container with id 9959f6bba4511696db174ee4102adbbc43b9c822aeaa20a11ed414860977ff12
Sep 16 16:41:07.155855 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:07.155822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" event={"ID":"c1287478-81eb-4b7c-a73f-89a7d26f296b","Type":"ContainerStarted","Data":"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"}
Sep 16 16:41:07.155959 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:07.155880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" event={"ID":"c1287478-81eb-4b7c-a73f-89a7d26f296b","Type":"ContainerStarted","Data":"9959f6bba4511696db174ee4102adbbc43b9c822aeaa20a11ed414860977ff12"}
Sep 16 16:41:07.155959 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:07.155948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:07.174477 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:07.174401    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podStartSLOduration=283.174365033 podStartE2EDuration="4m43.174365033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:07.17360707 +0000 UTC m=+284.259327141" watchObservedRunningTime="2026-09-16 16:41:07.174365033 +0000 UTC m=+284.260085107"
Sep 16 16:41:11.841065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:11.841034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:11.841458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:11.841099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:11.841458 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:11.841181    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:11.841458 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:11.841237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls podName:0b7b12f3-3483-4b5e-a990-efdbb2e82402 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:27.841224319 +0000 UTC m=+304.926944368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp2rq" (UID: "0b7b12f3-3483-4b5e-a990-efdbb2e82402") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:11.841572 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:11.841493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b7b12f3-3483-4b5e-a990-efdbb2e82402-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:15.385258 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:15.385210    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-z78gp" podUID="8a7b0509-a48f-420d-964c-caeebd79c4b8"
Sep 16 16:41:23.337513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:23.337480    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 16:41:26.759097 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:26.759052    2577 patch_prober.go:28] interesting pod/image-registry-6cf96db99c-l69ng container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 16:41:26.761484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:26.759112    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:41:27.536251 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.536224    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"]
Sep 16 16:41:27.539063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.539045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:27.541294 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.541268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 16 16:41:27.541456 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.541268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xcm4q\""
Sep 16 16:41:27.546869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.546849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"]
Sep 16 16:41:27.644796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.644762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ntlsd\" (UID: \"11ff97db-a382-45ca-af5a-ff918add216a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:27.745058 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.745036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ntlsd\" (UID: \"11ff97db-a382-45ca-af5a-ff918add216a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:27.745147 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:27.745121    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 16:41:27.745212 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:27.745166    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates podName:11ff97db-a382-45ca-af5a-ff918add216a nodeName:}" failed. No retries permitted until 2026-09-16 16:41:28.245154117 +0000 UTC m=+305.330874166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-ntlsd" (UID: "11ff97db-a382-45ca-af5a-ff918add216a") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 16:41:27.845833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.845780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:27.847899 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:27.847875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b7b12f3-3483-4b5e-a990-efdbb2e82402-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp2rq\" (UID: \"0b7b12f3-3483-4b5e-a990-efdbb2e82402\") " pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:28.139224 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.139173    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nnmt\""
Sep 16 16:41:28.146926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.146909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp2rq"
Sep 16 16:41:28.163920 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.163718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:41:28.249214 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.249128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ntlsd\" (UID: \"11ff97db-a382-45ca-af5a-ff918add216a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:28.250486 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:28.250414    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 16:41:28.250615 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:41:28.250592    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates podName:11ff97db-a382-45ca-af5a-ff918add216a nodeName:}" failed. No retries permitted until 2026-09-16 16:41:29.250493994 +0000 UTC m=+306.336214057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-ntlsd" (UID: "11ff97db-a382-45ca-af5a-ff918add216a") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 16:41:28.421646 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.421587    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp2rq"]
Sep 16 16:41:28.425884 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:28.425861    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7b12f3_3483_4b5e_a990_efdbb2e82402.slice/crio-df80fe2f8d3edd432934d9ad8f4cfea2c46687b64a26c94e4c03bcf7d1c2f5c8 WatchSource:0}: Error finding container df80fe2f8d3edd432934d9ad8f4cfea2c46687b64a26c94e4c03bcf7d1c2f5c8: Status 404 returned error can't find the container with id df80fe2f8d3edd432934d9ad8f4cfea2c46687b64a26c94e4c03bcf7d1c2f5c8
Sep 16 16:41:28.478265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:28.478249    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:41:29.227199 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.227162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp2rq" event={"ID":"0b7b12f3-3483-4b5e-a990-efdbb2e82402","Type":"ContainerStarted","Data":"67ccb18153553837898c44e7769c92e05ca1abbdcee30019aa31ef658d16d301"}
Sep 16 16:41:29.227617 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.227222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp2rq" event={"ID":"0b7b12f3-3483-4b5e-a990-efdbb2e82402","Type":"ContainerStarted","Data":"df80fe2f8d3edd432934d9ad8f4cfea2c46687b64a26c94e4c03bcf7d1c2f5c8"}
Sep 16 16:41:29.257585 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.257556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ntlsd\" (UID: \"11ff97db-a382-45ca-af5a-ff918add216a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:29.260157 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.260128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11ff97db-a382-45ca-af5a-ff918add216a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ntlsd\" (UID: \"11ff97db-a382-45ca-af5a-ff918add216a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:29.348431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.348404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:29.385369 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.385345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:41:29.629576 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:29.629524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"]
Sep 16 16:41:29.632654 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:29.632631    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ff97db_a382_45ca_af5a_ff918add216a.slice/crio-a21b99ca3187b43592662eaa9ff6dd8c8c35abb2cc95dc7d07ac7c04f2e89ba9 WatchSource:0}: Error finding container a21b99ca3187b43592662eaa9ff6dd8c8c35abb2cc95dc7d07ac7c04f2e89ba9: Status 404 returned error can't find the container with id a21b99ca3187b43592662eaa9ff6dd8c8c35abb2cc95dc7d07ac7c04f2e89ba9
Sep 16 16:41:30.232321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:30.232291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp2rq" event={"ID":"0b7b12f3-3483-4b5e-a990-efdbb2e82402","Type":"ContainerStarted","Data":"aaf2606a43f7f9042335a879fd20931a4adf70773acf617d4ecb70fbbcb9f34c"}
Sep 16 16:41:30.233559 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:30.233533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd" event={"ID":"11ff97db-a382-45ca-af5a-ff918add216a","Type":"ContainerStarted","Data":"a21b99ca3187b43592662eaa9ff6dd8c8c35abb2cc95dc7d07ac7c04f2e89ba9"}
Sep 16 16:41:32.242094 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.242057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd" event={"ID":"11ff97db-a382-45ca-af5a-ff918add216a","Type":"ContainerStarted","Data":"2cac5a752a0bc66c4b20529e263a1b1c7c392789515542794c6aab85f5642a49"}
Sep 16 16:41:32.242642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.242238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:32.245028 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.244984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp2rq" event={"ID":"0b7b12f3-3483-4b5e-a990-efdbb2e82402","Type":"ContainerStarted","Data":"bab0c37237e9c9351830c1e61e6e4fac3f007d4abda6992bc3e123e1c154c5b2"}
Sep 16 16:41:32.247772 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.247747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd"
Sep 16 16:41:32.259215 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.259172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ntlsd" podStartSLOduration=3.460463858 podStartE2EDuration="5.259160042s" totalImagesPullingTime="1.798696184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:32.258909932 +0000 UTC m=+309.344630005" watchObservedRunningTime="2026-09-16 16:41:32.259160042 +0000 UTC m=+309.344880105"
Sep 16 16:41:32.292742 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:32.292593    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xp2rq" podStartSLOduration=33.395673453 podStartE2EDuration="36.292577389s" totalImagesPullingTime="2.896903936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:40:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:32.292515553 +0000 UTC m=+309.378235636" watchObservedRunningTime="2026-09-16 16:41:32.292577389 +0000 UTC m=+309.378297462"
Sep 16 16:41:38.012291 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.012260    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"]
Sep 16 16:41:38.016653 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.016633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.016796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.016769    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"]
Sep 16 16:41:38.019534 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.019516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.025768 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.025749    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-677qz"]
Sep 16 16:41:38.026632 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.026610    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 16:41:38.027574 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.027554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 16:41:38.027697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.027591    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dw9tl\""
Sep 16 16:41:38.027697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.027600    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 16:41:38.027866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.027849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 16:41:38.028824 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.028804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.033779 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.033760    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 16:41:38.034087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.033767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 16:41:38.034200 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.033831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 16:41:38.034668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.034417    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 16:41:38.034668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.034539    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 16:41:38.034668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.034613    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 16:41:38.034861 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.034805    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 16:41:38.034913 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.034890    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-284lx\""
Sep 16 16:41:38.035192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.035175    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mn6wt\""
Sep 16 16:41:38.055292 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.055267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"]
Sep 16 16:41:38.057972 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.057950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"]
Sep 16 16:41:38.118325 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a086f-7f07-4325-a25f-99b769b8ca0f-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.118453 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-root\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118453 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-textfile\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118453 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.118596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.118596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-sys\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-wtmp\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-metrics-client-ca\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8tsw\" (UniqueName: \"kubernetes.io/projected/b729877b-df32-46d1-8389-5fd69da6ed1e-kube-api-access-n8tsw\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkx25\" (UniqueName: \"kubernetes.io/projected/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-api-access-mkx25\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft27n\" (UniqueName: \"kubernetes.io/projected/cc8a086f-7f07-4325-a25f-99b769b8ca0f-kube-api-access-ft27n\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.118817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.119076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cd87b230-c3f0-4fb0-bd3e-246153747821-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.119076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.119076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-tls\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.119076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.119076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.118943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.219357 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.219536 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.219536 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cd87b230-c3f0-4fb0-bd3e-246153747821-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.219648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.219648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-tls\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.219753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.219753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cd87b230-c3f0-4fb0-bd3e-246153747821-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.219753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.219753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a086f-7f07-4325-a25f-99b769b8ca0f-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.219753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-root\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-textfile\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-sys\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-wtmp\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-metrics-client-ca\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8tsw\" (UniqueName: \"kubernetes.io/projected/b729877b-df32-46d1-8389-5fd69da6ed1e-kube-api-access-n8tsw\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkx25\" (UniqueName: \"kubernetes.io/projected/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-api-access-mkx25\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft27n\" (UniqueName: \"kubernetes.io/projected/cc8a086f-7f07-4325-a25f-99b769b8ca0f-kube-api-access-ft27n\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.220004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.219995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.220079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.220564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.220162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-sys\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.220231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-root\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.220721 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.220663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a086f-7f07-4325-a25f-99b769b8ca0f-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.221053 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.221029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.221127 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.221092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b729877b-df32-46d1-8389-5fd69da6ed1e-metrics-client-ca\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.221199 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.221179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-wtmp\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.221984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.221964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-textfile\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.222809 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.222769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-tls\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.223151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.223098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.223431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.223411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b729877b-df32-46d1-8389-5fd69da6ed1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.223852 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.223831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.223944 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.223877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a086f-7f07-4325-a25f-99b769b8ca0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.224559 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.224535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.228887 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.228862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8tsw\" (UniqueName: \"kubernetes.io/projected/b729877b-df32-46d1-8389-5fd69da6ed1e-kube-api-access-n8tsw\") pod \"node-exporter-677qz\" (UID: \"b729877b-df32-46d1-8389-5fd69da6ed1e\") " pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.229823 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.229801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft27n\" (UniqueName: \"kubernetes.io/projected/cc8a086f-7f07-4325-a25f-99b769b8ca0f-kube-api-access-ft27n\") pod \"openshift-state-metrics-67dcf8c979-c88t6\" (UID: \"cc8a086f-7f07-4325-a25f-99b769b8ca0f\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.229914 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.229895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkx25\" (UniqueName: \"kubernetes.io/projected/cd87b230-c3f0-4fb0-bd3e-246153747821-kube-api-access-mkx25\") pod \"kube-state-metrics-59fbdd77d-2kn8d\" (UID: \"cd87b230-c3f0-4fb0-bd3e-246153747821\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.328488 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.328466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"
Sep 16 16:41:38.337191 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.337167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"
Sep 16 16:41:38.342896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.342877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-677qz"
Sep 16 16:41:38.355281 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:38.355258    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb729877b_df32_46d1_8389_5fd69da6ed1e.slice/crio-5a4235256e7931a274a245fe664170deafb4fae45c754add85ecf83a77f1a21a WatchSource:0}: Error finding container 5a4235256e7931a274a245fe664170deafb4fae45c754add85ecf83a77f1a21a: Status 404 returned error can't find the container with id 5a4235256e7931a274a245fe664170deafb4fae45c754add85ecf83a77f1a21a
Sep 16 16:41:38.608172 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.608111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6"]
Sep 16 16:41:38.610924 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:38.610890    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8a086f_7f07_4325_a25f_99b769b8ca0f.slice/crio-74e84df263ae5d27fb2a9fd341e5d94558f626cedd89fa97952cef6767b15900 WatchSource:0}: Error finding container 74e84df263ae5d27fb2a9fd341e5d94558f626cedd89fa97952cef6767b15900: Status 404 returned error can't find the container with id 74e84df263ae5d27fb2a9fd341e5d94558f626cedd89fa97952cef6767b15900
Sep 16 16:41:38.614715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:38.614694    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d"]
Sep 16 16:41:38.617037 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:38.617018    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd87b230_c3f0_4fb0_bd3e_246153747821.slice/crio-8537511b51f289beda946e215a25fcd9a6abcf123ce0d303baa629e9ddb45ad3 WatchSource:0}: Error finding container 8537511b51f289beda946e215a25fcd9a6abcf123ce0d303baa629e9ddb45ad3: Status 404 returned error can't find the container with id 8537511b51f289beda946e215a25fcd9a6abcf123ce0d303baa629e9ddb45ad3
Sep 16 16:41:39.270639 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:39.270605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6" event={"ID":"cc8a086f-7f07-4325-a25f-99b769b8ca0f","Type":"ContainerStarted","Data":"2143a0df89da961729d6607793fdc80b0cc08e55ced658980a17972d9616a6c7"}
Sep 16 16:41:39.271042 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:39.270673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6" event={"ID":"cc8a086f-7f07-4325-a25f-99b769b8ca0f","Type":"ContainerStarted","Data":"c8e3b07b29b2522fc1e8bac3f9cbddd277a815761c651c4401be728d2e07b477"}
Sep 16 16:41:39.271042 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:39.270703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6" event={"ID":"cc8a086f-7f07-4325-a25f-99b769b8ca0f","Type":"ContainerStarted","Data":"74e84df263ae5d27fb2a9fd341e5d94558f626cedd89fa97952cef6767b15900"}
Sep 16 16:41:39.272049 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:39.272008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-677qz" event={"ID":"b729877b-df32-46d1-8389-5fd69da6ed1e","Type":"ContainerStarted","Data":"5a4235256e7931a274a245fe664170deafb4fae45c754add85ecf83a77f1a21a"}
Sep 16 16:41:39.273438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:39.273412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d" event={"ID":"cd87b230-c3f0-4fb0-bd3e-246153747821","Type":"ContainerStarted","Data":"8537511b51f289beda946e215a25fcd9a6abcf123ce0d303baa629e9ddb45ad3"}
Sep 16 16:41:40.280150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:40.280113    2577 generic.go:356] "Generic (PLEG): container finished" podID="b729877b-df32-46d1-8389-5fd69da6ed1e" containerID="54e75c6a46ed1571b1b6784a4b9345986c1c74da52c4e0726b16e1af19770512" exitCode=0
Sep 16 16:41:40.280595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:40.280185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-677qz" event={"ID":"b729877b-df32-46d1-8389-5fd69da6ed1e","Type":"ContainerDied","Data":"54e75c6a46ed1571b1b6784a4b9345986c1c74da52c4e0726b16e1af19770512"}
Sep 16 16:41:41.289449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.289411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-677qz" event={"ID":"b729877b-df32-46d1-8389-5fd69da6ed1e","Type":"ContainerStarted","Data":"f44fa011835270dacc65280ae1db89167274b5acba0cda1ed182d38da356b212"}
Sep 16 16:41:41.289856 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.289499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-677qz" event={"ID":"b729877b-df32-46d1-8389-5fd69da6ed1e","Type":"ContainerStarted","Data":"afea2a5aa0e8fcd66d275eb19dad3b835ac0fa9b82f8e548f20de839745b83b2"}
Sep 16 16:41:41.291693 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.291664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d" event={"ID":"cd87b230-c3f0-4fb0-bd3e-246153747821","Type":"ContainerStarted","Data":"e9564bf3668f2b950edb904ebd558be92e5cff3a1b66647a96d9f853fd9b7d90"}
Sep 16 16:41:41.291898 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.291720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d" event={"ID":"cd87b230-c3f0-4fb0-bd3e-246153747821","Type":"ContainerStarted","Data":"a30d3e4b2b70ef8874360a2a60c31b35449fd08425895aef0fb40c49a0cac076"}
Sep 16 16:41:41.291898 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.291749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d" event={"ID":"cd87b230-c3f0-4fb0-bd3e-246153747821","Type":"ContainerStarted","Data":"e6bb8428b11a08774cbfd312246c39859c6c31e510362917b95f74ac341605a3"}
Sep 16 16:41:41.293795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.293770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6" event={"ID":"cc8a086f-7f07-4325-a25f-99b769b8ca0f","Type":"ContainerStarted","Data":"345a00772b0fe4f773c6b385a7ba3260518a7e31d90bf52aa910acb472fdcfce"}
Sep 16 16:41:41.308591 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.308549    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-677qz" podStartSLOduration=3.149499677 podStartE2EDuration="4.308536235s" totalImagesPullingTime="1.099503654s" totalInitContainerRuntime="59.532904ms" isStatefulPod=true podCreationTimestamp="2026-09-16 16:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:41.307071502 +0000 UTC m=+318.392791575" watchObservedRunningTime="2026-09-16 16:41:41.308536235 +0000 UTC m=+318.394256307"
Sep 16 16:41:41.326335 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.326288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-c88t6" podStartSLOduration=2.638679506 podStartE2EDuration="4.326273927s" totalImagesPullingTime="1.687594421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:41.325248594 +0000 UTC m=+318.410968665" watchObservedRunningTime="2026-09-16 16:41:41.326273927 +0000 UTC m=+318.411994056"
Sep 16 16:41:41.347186 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:41.347147    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2kn8d" podStartSLOduration=2.552505167 podStartE2EDuration="4.347132909s" totalImagesPullingTime="1.794627742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:41.34649379 +0000 UTC m=+318.432213937" watchObservedRunningTime="2026-09-16 16:41:41.347132909 +0000 UTC m=+318.432852980"
Sep 16 16:41:42.485599 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.485565    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5dc4665f77-5nhqh"]
Sep 16 16:41:42.488782 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.488761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.491132 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nw4r8\""
Sep 16 16:41:42.491261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 16:41:42.491843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491824    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 16:41:42.491843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491834    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1rcuipr8aq9hu\""
Sep 16 16:41:42.491981 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 16:41:42.491981 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.491847    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 16:41:42.499484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.499463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc4665f77-5nhqh"]
Sep 16 16:41:42.554771 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.554746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-metrics-server-audit-profiles\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.554872 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.554777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-client-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.554872 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.554801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-client-certs\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.554982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.554917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncvn2\" (UniqueName: \"kubernetes.io/projected/1fd729a5-2884-47d9-a384-f5dd56f221f3-kube-api-access-ncvn2\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.555031 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.555018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fd729a5-2884-47d9-a384-f5dd56f221f3-audit-log\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.555089 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.555049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.555150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.555091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-tls\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656239 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncvn2\" (UniqueName: \"kubernetes.io/projected/1fd729a5-2884-47d9-a384-f5dd56f221f3-kube-api-access-ncvn2\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fd729a5-2884-47d9-a384-f5dd56f221f3-audit-log\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-tls\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-metrics-server-audit-profiles\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656614 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-client-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656614 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-client-certs\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.656763 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.656740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fd729a5-2884-47d9-a384-f5dd56f221f3-audit-log\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.657237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.657214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.657516 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.657489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fd729a5-2884-47d9-a384-f5dd56f221f3-metrics-server-audit-profiles\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.659149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.659113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-tls\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.659260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.659189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-client-ca-bundle\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.659260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.659241    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fd729a5-2884-47d9-a384-f5dd56f221f3-secret-metrics-server-client-certs\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.663817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.663791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncvn2\" (UniqueName: \"kubernetes.io/projected/1fd729a5-2884-47d9-a384-f5dd56f221f3-kube-api-access-ncvn2\") pod \"metrics-server-5dc4665f77-5nhqh\" (UID: \"1fd729a5-2884-47d9-a384-f5dd56f221f3\") " pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.798633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.798606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:41:42.979589 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:42.979561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:41:43.068344 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:43.068303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc4665f77-5nhqh"]
Sep 16 16:41:43.070377 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:43.070350    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd729a5_2884_47d9_a384_f5dd56f221f3.slice/crio-d2059383621ea2da816621cfbe7cbcee964af9b34b437237fb97bb2b5fb3e4aa WatchSource:0}: Error finding container d2059383621ea2da816621cfbe7cbcee964af9b34b437237fb97bb2b5fb3e4aa: Status 404 returned error can't find the container with id d2059383621ea2da816621cfbe7cbcee964af9b34b437237fb97bb2b5fb3e4aa
Sep 16 16:41:43.302243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:43.302222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh" event={"ID":"1fd729a5-2884-47d9-a384-f5dd56f221f3","Type":"ContainerStarted","Data":"d2059383621ea2da816621cfbe7cbcee964af9b34b437237fb97bb2b5fb3e4aa"}
Sep 16 16:41:44.264934 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.264900    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:41:44.270244 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.270224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.272782 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.272663    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7mj05chh8kocs\""
Sep 16 16:41:44.272894 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.272692    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 16:41:44.272894 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.272750    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vz4s4\""
Sep 16 16:41:44.272894 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.272888    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 16:41:44.273061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.272775    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273647    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273670    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273683    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273670    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 16:41:44.273811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.273766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 16:41:44.274486 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.274467    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 16:41:44.274919 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.274888    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 16:41:44.275246 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.275227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 16:41:44.275471 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.275453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 16:41:44.277625 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.277596    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 16:41:44.287246 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.287229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:41:44.371047 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5spq\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.371884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.371874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473024 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.472998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473437 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473437 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473437 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5spq\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.473810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.473791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.474599 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.474571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.475719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.475691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.476695 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.476666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.476957 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.476694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.476957 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.476731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.476957 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.476918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.477170 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.477709 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.477709 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.477861 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.477952 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.478014 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.477986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.478724 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.478697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.478827 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.478756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.479222 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.479194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.479966 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.479944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.480180 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.480163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.481439 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.481423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5spq\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq\") pod \"prometheus-k8s-0\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.582018 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.581991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:41:44.863234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:44.863209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:41:44.866148 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:41:44.866124    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568937d7_941f_4dea_83f1_deaaac524170.slice/crio-e2ad01363264a54589476db8e32bf9765c47ec10fb00508f762637e2da168575 WatchSource:0}: Error finding container e2ad01363264a54589476db8e32bf9765c47ec10fb00508f762637e2da168575: Status 404 returned error can't find the container with id e2ad01363264a54589476db8e32bf9765c47ec10fb00508f762637e2da168575
Sep 16 16:41:45.311906 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:45.311866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh" event={"ID":"1fd729a5-2884-47d9-a384-f5dd56f221f3","Type":"ContainerStarted","Data":"7e8dc374f4d12d787581db931b889bfda72a5138ba7465ae0408e021c7bfa331"}
Sep 16 16:41:45.313517 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:45.313491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"e2ad01363264a54589476db8e32bf9765c47ec10fb00508f762637e2da168575"}
Sep 16 16:41:45.336286 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:45.331701    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh" podStartSLOduration=1.918534126 podStartE2EDuration="3.331686032s" totalImagesPullingTime="1.413151906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:45.330102457 +0000 UTC m=+322.415822534" watchObservedRunningTime="2026-09-16 16:41:45.331686032 +0000 UTC m=+322.417406104"
Sep 16 16:41:46.318588 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:46.318555    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a" exitCode=0
Sep 16 16:41:46.318992 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:46.318658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"}
Sep 16 16:41:50.340061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:50.340029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"}
Sep 16 16:41:50.340472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:50.340145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"}
Sep 16 16:41:52.350735 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:52.350691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"}
Sep 16 16:41:52.351167 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:52.350774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"}
Sep 16 16:41:52.351167 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:52.350802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"}
Sep 16 16:41:52.351167 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:52.350827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerStarted","Data":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"}
Sep 16 16:41:52.384148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:52.384105    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.969109279 podStartE2EDuration="8.384090147s" totalImagesPullingTime="6.393904337s" totalInitContainerRuntime="21.076531ms" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:52.381975725 +0000 UTC m=+329.467695797" watchObservedRunningTime="2026-09-16 16:41:52.384090147 +0000 UTC m=+329.469810218"
Sep 16 16:41:54.582452 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:41:54.582418    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:42:02.799239 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:02.799205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:42:02.799675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:02.799268    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:42:07.998313 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:07.998258    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry" containerID="cri-o://636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2" gracePeriod=30
Sep 16 16:42:08.159482 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.159452    2577 patch_prober.go:28] interesting pod/image-registry-6cf96db99c-l69ng container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body=
Sep 16 16:42:08.159648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.159504    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused"
Sep 16 16:42:08.245964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.245942    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:42:08.264121 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264121 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264160    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264177    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jncsv\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264232    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264533 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token\") pod \"c1287478-81eb-4b7c-a73f-89a7d26f296b\" (UID: \"c1287478-81eb-4b7c-a73f-89a7d26f296b\") "
Sep 16 16:42:08.264806 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264713    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:08.264806 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.264732    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:08.267662 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.267631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv" (OuterVolumeSpecName: "kube-api-access-jncsv") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "kube-api-access-jncsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:08.267772 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.267707    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:08.267837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.267805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:08.268069 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.268037    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:08.268069 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.268057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:08.275833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.275805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c1287478-81eb-4b7c-a73f-89a7d26f296b" (UID: "c1287478-81eb-4b7c-a73f-89a7d26f296b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:08.365595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365571    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-image-registry-private-configuration\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365593    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365602    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jncsv\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-kube-api-access-jncsv\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365611    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1287478-81eb-4b7c-a73f-89a7d26f296b-ca-trust-extracted\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365621    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-registry-certificates\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365630    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1287478-81eb-4b7c-a73f-89a7d26f296b-bound-sa-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365638    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1287478-81eb-4b7c-a73f-89a7d26f296b-installation-pull-secrets\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.365715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.365647    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1287478-81eb-4b7c-a73f-89a7d26f296b-trusted-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:42:08.409099 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.409072    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerID="636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2" exitCode=0
Sep 16 16:42:08.409180 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.409112    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng"
Sep 16 16:42:08.409223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.409112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" event={"ID":"c1287478-81eb-4b7c-a73f-89a7d26f296b","Type":"ContainerDied","Data":"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"}
Sep 16 16:42:08.409223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.409212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf96db99c-l69ng" event={"ID":"c1287478-81eb-4b7c-a73f-89a7d26f296b","Type":"ContainerDied","Data":"9959f6bba4511696db174ee4102adbbc43b9c822aeaa20a11ed414860977ff12"}
Sep 16 16:42:08.409285 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.409251    2577 scope.go:117] "RemoveContainer" containerID="636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"
Sep 16 16:42:08.418345 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.418323    2577 scope.go:117] "RemoveContainer" containerID="636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"
Sep 16 16:42:08.418665 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:42:08.418645    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2\": container with ID starting with 636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2 not found: ID does not exist" containerID="636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"
Sep 16 16:42:08.418737 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.418677    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2"} err="failed to get container status \"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2\": rpc error: code = NotFound desc = could not find container \"636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2\": container with ID starting with 636ac6dc2680d356b884e1f038d9d597ff0294b121e666063187139f82e8bff2 not found: ID does not exist"
Sep 16 16:42:08.430693 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.430669    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:42:08.434090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:08.434070    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cf96db99c-l69ng"]
Sep 16 16:42:09.390040 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:09.390011    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" path="/var/lib/kubelet/pods/c1287478-81eb-4b7c-a73f-89a7d26f296b/volumes"
Sep 16 16:42:16.438307 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:16.438237    2577 generic.go:356] "Generic (PLEG): container finished" podID="8fd0d9db-5946-4d2e-9da3-3d0fe8246520" containerID="6ae0ea90cb6358f9d45807708f1d1da21ea1603e5df4960f8580328656fedd65" exitCode=0
Sep 16 16:42:16.438630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:16.438321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-25qmb" event={"ID":"8fd0d9db-5946-4d2e-9da3-3d0fe8246520","Type":"ContainerDied","Data":"6ae0ea90cb6358f9d45807708f1d1da21ea1603e5df4960f8580328656fedd65"}
Sep 16 16:42:16.438680 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:16.438653    2577 scope.go:117] "RemoveContainer" containerID="6ae0ea90cb6358f9d45807708f1d1da21ea1603e5df4960f8580328656fedd65"
Sep 16 16:42:17.444523 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:17.444490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-25qmb" event={"ID":"8fd0d9db-5946-4d2e-9da3-3d0fe8246520","Type":"ContainerStarted","Data":"d56eaf1c10505e32be0ba283c49f858277497d5bbc16f971fc15d5de2021498e"}
Sep 16 16:42:22.804299 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:22.804272    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:42:22.808176 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:22.808155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5dc4665f77-5nhqh"
Sep 16 16:42:37.189077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:37.189032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:42:37.191615 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:37.191584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a7b0509-a48f-420d-964c-caeebd79c4b8-metrics-certs\") pod \"network-metrics-daemon-z78gp\" (UID: \"8a7b0509-a48f-420d-964c-caeebd79c4b8\") " pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:42:37.488886 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:37.488812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6lts\""
Sep 16 16:42:37.496871 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:37.496846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z78gp"
Sep 16 16:42:37.770072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:37.770051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z78gp"]
Sep 16 16:42:37.773054 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:42:37.773029    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7b0509_a48f_420d_964c_caeebd79c4b8.slice/crio-a80e04c10df8565710c3a93c58ca9ef3d684f973ed193ce1f85807acc9d1f234 WatchSource:0}: Error finding container a80e04c10df8565710c3a93c58ca9ef3d684f973ed193ce1f85807acc9d1f234: Status 404 returned error can't find the container with id a80e04c10df8565710c3a93c58ca9ef3d684f973ed193ce1f85807acc9d1f234
Sep 16 16:42:38.522436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:38.522395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z78gp" event={"ID":"8a7b0509-a48f-420d-964c-caeebd79c4b8","Type":"ContainerStarted","Data":"a80e04c10df8565710c3a93c58ca9ef3d684f973ed193ce1f85807acc9d1f234"}
Sep 16 16:42:40.532818 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:40.532773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z78gp" event={"ID":"8a7b0509-a48f-420d-964c-caeebd79c4b8","Type":"ContainerStarted","Data":"93ece92a72565f86166c54822913fec6a75263937b90a650c1e34421cd4779b7"}
Sep 16 16:42:40.533303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:40.532841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z78gp" event={"ID":"8a7b0509-a48f-420d-964c-caeebd79c4b8","Type":"ContainerStarted","Data":"1d223e4f39f67a9a24deca31b0279f533908fd84f16d15d6f8c970886a0f67cb"}
Sep 16 16:42:40.549572 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:40.549520    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z78gp" podStartSLOduration=375.87724468 podStartE2EDuration="6m17.549502598s" totalImagesPullingTime="1.672257918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:42:40.54879408 +0000 UTC m=+377.634514147" watchObservedRunningTime="2026-09-16 16:42:40.549502598 +0000 UTC m=+377.635222701"
Sep 16 16:42:44.582409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:44.582367    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:42:44.600967 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:44.600945    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:42:45.568984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:42:45.568953    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:02.664033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.663992    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:02.664464 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664311    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="prometheus" containerID="cri-o://6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04" gracePeriod=600
Sep 16 16:43:02.664464 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664354    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy" containerID="cri-o://56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8" gracePeriod=600
Sep 16 16:43:02.664464 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664374    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="config-reloader" containerID="cri-o://567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602" gracePeriod=600
Sep 16 16:43:02.664464 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664410    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776" gracePeriod=600
Sep 16 16:43:02.664663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664410    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-web" containerID="cri-o://2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8" gracePeriod=600
Sep 16 16:43:02.664663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.664332    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="thanos-sidecar" containerID="cri-o://f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980" gracePeriod=600
Sep 16 16:43:02.903719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.903689    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:02.996863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996826    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.996863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996871    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996914    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.996988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997019    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5spq\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997113    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997167    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997237    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.997666 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db\") pod \"568937d7-941f-4dea-83f1-deaaac524170\" (UID: \"568937d7-941f-4dea-83f1-deaaac524170\") "
Sep 16 16:43:02.998119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:02.998119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997513    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:02.998119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997509    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:02.998119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.997948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:02.998410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:02.998362    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:03.000133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000026    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.000133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:03.000133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000103    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.000509 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000481    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.000899 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000860    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:03.000994 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out" (OuterVolumeSpecName: "config-out") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:03.000994 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000975    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.000994 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.000983    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq" (OuterVolumeSpecName: "kube-api-access-j5spq") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "kube-api-access-j5spq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:43:03.001186 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.001079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.002022 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.001986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:03.002601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.002578    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:43:03.002750 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.002732    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config" (OuterVolumeSpecName: "config") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.002808 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.002748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.002911 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.002894    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.011538 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.011496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config" (OuterVolumeSpecName: "web-config") pod "568937d7-941f-4dea-83f1-deaaac524170" (UID: "568937d7-941f-4dea-83f1-deaaac524170"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:03.098708 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098679    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098708 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098707    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-grpc-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098716    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-tls-assets\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098727    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098737    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-db\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098745    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-kube-rbac-proxy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098754    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098762    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098771    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-metrics-client-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098781    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098789    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098798    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098806    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5spq\" (UniqueName: \"kubernetes.io/projected/568937d7-941f-4dea-83f1-deaaac524170-kube-api-access-j5spq\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098820    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098834    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-web-config\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098843    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098852    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/568937d7-941f-4dea-83f1-deaaac524170-config-out\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098860    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/568937d7-941f-4dea-83f1-deaaac524170-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098868    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-secret-metrics-client-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.098908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.098876    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/568937d7-941f-4dea-83f1-deaaac524170-config\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:43:03.626071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626029    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776" exitCode=0
Sep 16 16:43:03.626071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626063    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8" exitCode=0
Sep 16 16:43:03.626071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626071    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8" exitCode=0
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626082    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980" exitCode=0
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626091    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602" exitCode=0
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626099    2577 generic.go:356] "Generic (PLEG): container finished" podID="568937d7-941f-4dea-83f1-deaaac524170" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04" exitCode=0
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"}
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"}
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"}
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"}
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626266    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"}
Sep 16 16:43:03.626305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"}
Sep 16 16:43:03.626645 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626312    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.626645 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.626317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"568937d7-941f-4dea-83f1-deaaac524170","Type":"ContainerDied","Data":"e2ad01363264a54589476db8e32bf9765c47ec10fb00508f762637e2da168575"}
Sep 16 16:43:03.634340 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.634318    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.642052 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.642024    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.645704 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.645676    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:03.649655 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.649593    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:03.651080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.651063    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.658730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.658707    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.666828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.666668    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.674638 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674608    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:03.674886 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674874    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674888    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674898    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="config-reloader"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674904    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="config-reloader"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674913    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-web"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674918    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-web"
Sep 16 16:43:03.674926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674924    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="prometheus"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674930    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="prometheus"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674941    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-thanos"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674946    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-thanos"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674953    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="init-config-reloader"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674958    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="init-config-reloader"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674968    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="thanos-sidecar"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="thanos-sidecar"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674980    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.674985    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675024    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-thanos"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675031    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="config-reloader"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675036    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="thanos-sidecar"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675044    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="prometheus"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675050    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675056    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1287478-81eb-4b7c-a73f-89a7d26f296b" containerName="registry"
Sep 16 16:43:03.675096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675061    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="568937d7-941f-4dea-83f1-deaaac524170" containerName="kube-rbac-proxy-web"
Sep 16 16:43:03.675659 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.675285    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.680194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.680169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.682704 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.682675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 16:43:03.682847 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.682713    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 16:43:03.682847 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.682675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 16:43:03.682847 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.682675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 16:43:03.682847 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.682675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 16:43:03.683141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 16:43:03.683141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683133    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vz4s4\""
Sep 16 16:43:03.683265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683131    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 16:43:03.683265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 16:43:03.683373 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 16:43:03.683489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683471    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 16:43:03.683489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683486    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 16:43:03.683590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683555    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7mj05chh8kocs\""
Sep 16 16:43:03.683772 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.683758    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 16:43:03.684649 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.684629    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.685121 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.685006    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.685121 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685054    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.685121 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685074    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.685560 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.685426    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.685560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685456    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.685560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685488    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.685875 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.685782    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.685875 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685815    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.685875 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.685837    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.686202 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.686083    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.686202 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.686119    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.686202 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.686133    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.687042 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.686972    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.687042 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.687009    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.687150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.687138    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.687515 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.687485    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.687619 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.687523    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.687619 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.687547    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.692556 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.692523    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 16:43:03.694312 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:03.694277    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.694463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.694325    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.694463 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.694352    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.694961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.694929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.695056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.694965    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.695882 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.695849    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.696003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.695884    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.696441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.696369    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.696628 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.696562    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.696868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.696848    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 16:43:03.696950 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.696879    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.696950 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.696902    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.697245 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:03.697345 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697288    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.697345 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697310    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.697670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697633    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.697728 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697672    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.697962 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697942    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.698036 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.697964    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.698266 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698246    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.698318 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698268    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.698568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698546    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.698624 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698570    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.698868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698849    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.698951 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.698869    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.699181 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.699156    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.699181 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.699181    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.699634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.699600    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.699736 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.699636    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.700043 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700018    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.700104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700046    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.700323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700303    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.700407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700323    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.700590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700568    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.700590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700588    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.700888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700854    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.700888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.700879    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.701149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701132    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.701149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701149    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.701371 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701352    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.701371 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701371    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.701663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701644    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.701663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701662    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.701900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701884    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.701900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.701898    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.702137 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702118    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.702137 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702134    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.702359 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702342    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.702359 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702358    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.702642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702621    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.702714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702644    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.702896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702875    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.702958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.702898    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.703184 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703167    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.703253 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703187    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.703456 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703435    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.703537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703457    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.703702 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703686    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.703769 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703703    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.703943 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703926    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.703989 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.703944    2577 scope.go:117] "RemoveContainer" containerID="2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"
Sep 16 16:43:03.704184 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704166    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776"} err="failed to get container status \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": rpc error: code = NotFound desc = could not find container \"2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776\": container with ID starting with 2a2ec4e30f87d52dd204330b522bbe4a58d12b9523081d6df978f816e0f64776 not found: ID does not exist"
Sep 16 16:43:03.704234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704185    2577 scope.go:117] "RemoveContainer" containerID="56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"
Sep 16 16:43:03.704449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704430    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8"} err="failed to get container status \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": rpc error: code = NotFound desc = could not find container \"56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8\": container with ID starting with 56f8d01d762fc0603cb9610e750fac9a5a0052aa5c98f17ca6edc2d73eb16ee8 not found: ID does not exist"
Sep 16 16:43:03.704449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704448    2577 scope.go:117] "RemoveContainer" containerID="2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"
Sep 16 16:43:03.704699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704681    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8"} err="failed to get container status \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": rpc error: code = NotFound desc = could not find container \"2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8\": container with ID starting with 2aa8173b0efb75b7e9c3474fbfe2eb408a036c9e673c0140b2965722e78677b8 not found: ID does not exist"
Sep 16 16:43:03.704699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704697    2577 scope.go:117] "RemoveContainer" containerID="f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"
Sep 16 16:43:03.704949 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980"} err="failed to get container status \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": rpc error: code = NotFound desc = could not find container \"f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980\": container with ID starting with f7b13579575acdb2d46312fc717cdf1afc415f30e43cb6336beebfb76756d980 not found: ID does not exist"
Sep 16 16:43:03.705027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.704951    2577 scope.go:117] "RemoveContainer" containerID="567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"
Sep 16 16:43:03.705179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.705162    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602"} err="failed to get container status \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": rpc error: code = NotFound desc = could not find container \"567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602\": container with ID starting with 567c89f4b2f2653679828fb1ad60f1d8c33ecfeedd55d369a98d5025b088f602 not found: ID does not exist"
Sep 16 16:43:03.705249 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.705180    2577 scope.go:117] "RemoveContainer" containerID="6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"
Sep 16 16:43:03.705415 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.705395    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04"} err="failed to get container status \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": rpc error: code = NotFound desc = could not find container \"6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04\": container with ID starting with 6d794e9656c65c99330e88d6eb28e56aa3d28dd7d7c266403a4d7d5fd003ee04 not found: ID does not exist"
Sep 16 16:43:03.705494 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.705417    2577 scope.go:117] "RemoveContainer" containerID="279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"
Sep 16 16:43:03.705633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.705616    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a"} err="failed to get container status \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": rpc error: code = NotFound desc = could not find container \"279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a\": container with ID starting with 279b990f0c6d6b745d2c34027ce3b6a8908d6cc243d6fa2ec9a6ee6839e9ae2a not found: ID does not exist"
Sep 16 16:43:03.804179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804372 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804372 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804372 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-web-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbrpp\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-kube-api-access-dbrpp\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-config-out\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.804944 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.804769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbrpp\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-kube-api-access-dbrpp\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-config-out\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.905958 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.905996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-web-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.906843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.906817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.907098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.907072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.908944 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.908908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/919737a7-e673-4625-91f6-e2afc6a68852-config-out\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.908944 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.908909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.909227 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.909200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.909310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.909232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.909310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.909250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911091 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/919737a7-e673-4625-91f6-e2afc6a68852-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911282 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-web-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911282 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-config\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911519 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911685 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.911963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.911878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/919737a7-e673-4625-91f6-e2afc6a68852-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:03.913293 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:03.913271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbrpp\" (UniqueName: \"kubernetes.io/projected/919737a7-e673-4625-91f6-e2afc6a68852-kube-api-access-dbrpp\") pod \"prometheus-k8s-0\" (UID: \"919737a7-e673-4625-91f6-e2afc6a68852\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:04.012186 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:04.012145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:04.295257 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:04.295143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 16:43:04.297851 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:43:04.297825    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919737a7_e673_4625_91f6_e2afc6a68852.slice/crio-95091b7e6b1f9f3d38ddcf1504dd5874a0342817adc0a5fb98e64f54b1aebc1d WatchSource:0}: Error finding container 95091b7e6b1f9f3d38ddcf1504dd5874a0342817adc0a5fb98e64f54b1aebc1d: Status 404 returned error can't find the container with id 95091b7e6b1f9f3d38ddcf1504dd5874a0342817adc0a5fb98e64f54b1aebc1d
Sep 16 16:43:04.633105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:04.633018    2577 generic.go:356] "Generic (PLEG): container finished" podID="919737a7-e673-4625-91f6-e2afc6a68852" containerID="e0a69ba77baf1a3e08bfe5fb7ddfa618972d9e36561cf21dff8407d754858626" exitCode=0
Sep 16 16:43:04.633105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:04.633051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerDied","Data":"e0a69ba77baf1a3e08bfe5fb7ddfa618972d9e36561cf21dff8407d754858626"}
Sep 16 16:43:04.633291 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:04.633121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"95091b7e6b1f9f3d38ddcf1504dd5874a0342817adc0a5fb98e64f54b1aebc1d"}
Sep 16 16:43:05.390165 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.390126    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="568937d7-941f-4dea-83f1-deaaac524170" path="/var/lib/kubelet/pods/568937d7-941f-4dea-83f1-deaaac524170/volumes"
Sep 16 16:43:05.644075 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.643981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"bdcf93078a454fbdfa198743a28ba1f8b313d8331cbc5053bcf94bde444cae8f"}
Sep 16 16:43:05.644075 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.644046    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"58f2ffebb79667f7a0134b005258d399930c17b8693f76645558fd29e30abbcf"}
Sep 16 16:43:05.644075 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.644077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"a1007c0776992c39585bcd091890de00052a6848905e0228e06f973a0b432f9f"}
Sep 16 16:43:05.644336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.644103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"55aaf87062681458148d0edb6d86418b89bcf9213737f178e1b4d2821dd2ecb0"}
Sep 16 16:43:05.644336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.644126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"c1543c30b291df3674f6658587a9386fd266b6da5e51106372cf0122ee4e6001"}
Sep 16 16:43:05.644336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.644151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"919737a7-e673-4625-91f6-e2afc6a68852","Type":"ContainerStarted","Data":"15249f6d6cd7ea5a3f0bbb4c65b112a405895192d9aee7af907c1b5584026842"}
Sep 16 16:43:05.673130 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:05.672910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.655292443 podStartE2EDuration="2.672880898s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.588455ms" isStatefulPod=false podCreationTimestamp="2026-09-16 16:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:43:05.669367998 +0000 UTC m=+402.755088070" watchObservedRunningTime="2026-09-16 16:43:05.672880898 +0000 UTC m=+402.758600970"
Sep 16 16:43:06.143215 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:06.143172    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" podUID="86791c8b-0f46-4f45-a713-7f8dfe9d78fa"
Sep 16 16:43:06.648761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:06.648730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:43:07.145911 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:43:07.145866    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b55tq" podUID="87ecf712-f78e-4aca-84be-6a24fae7c992"
Sep 16 16:43:07.652983 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:07.652946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:43:08.853081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:08.853039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:43:08.853081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:08.853079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:43:08.855505 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:08.855482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87ecf712-f78e-4aca-84be-6a24fae7c992-metrics-tls\") pod \"dns-default-b55tq\" (UID: \"87ecf712-f78e-4aca-84be-6a24fae7c992\") " pod="openshift-dns/dns-default-b55tq"
Sep 16 16:43:08.855632 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:08.855614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86791c8b-0f46-4f45-a713-7f8dfe9d78fa-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-smsn2\" (UID: \"86791c8b-0f46-4f45-a713-7f8dfe9d78fa\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:43:09.012558 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.012517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:43:09.051739 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.051712    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v2gqr\""
Sep 16 16:43:09.059995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.059966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"
Sep 16 16:43:09.155813 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.155782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqrg9\""
Sep 16 16:43:09.163697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.163654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:43:09.334343 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.334302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2"]
Sep 16 16:43:09.337228 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:43:09.337200    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86791c8b_0f46_4f45_a713_7f8dfe9d78fa.slice/crio-b11c95e230a36e2994341e96e3174e2d2f6fe40cb4957acfaf4c046153406517 WatchSource:0}: Error finding container b11c95e230a36e2994341e96e3174e2d2f6fe40cb4957acfaf4c046153406517: Status 404 returned error can't find the container with id b11c95e230a36e2994341e96e3174e2d2f6fe40cb4957acfaf4c046153406517
Sep 16 16:43:09.436953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.436865    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b55tq"]
Sep 16 16:43:09.439702 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:43:09.439674    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ecf712_f78e_4aca_84be_6a24fae7c992.slice/crio-b8ce4647b6bd8e679aa19ebf416d0cf790e29d6defe3f801b4fcc8512b2ff3e7 WatchSource:0}: Error finding container b8ce4647b6bd8e679aa19ebf416d0cf790e29d6defe3f801b4fcc8512b2ff3e7: Status 404 returned error can't find the container with id b8ce4647b6bd8e679aa19ebf416d0cf790e29d6defe3f801b4fcc8512b2ff3e7
Sep 16 16:43:09.661799 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.661760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" event={"ID":"86791c8b-0f46-4f45-a713-7f8dfe9d78fa","Type":"ContainerStarted","Data":"b11c95e230a36e2994341e96e3174e2d2f6fe40cb4957acfaf4c046153406517"}
Sep 16 16:43:09.662963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:09.662935    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b55tq" event={"ID":"87ecf712-f78e-4aca-84be-6a24fae7c992","Type":"ContainerStarted","Data":"b8ce4647b6bd8e679aa19ebf416d0cf790e29d6defe3f801b4fcc8512b2ff3e7"}
Sep 16 16:43:11.672449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.672332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b55tq" event={"ID":"87ecf712-f78e-4aca-84be-6a24fae7c992","Type":"ContainerStarted","Data":"6556a474bb46e6f33a8a19cb66f534e425a0a411d01bd70dc0b36d05b607518e"}
Sep 16 16:43:11.672449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.672448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b55tq" event={"ID":"87ecf712-f78e-4aca-84be-6a24fae7c992","Type":"ContainerStarted","Data":"db69c55aac74b37501b0d1f66d987dc0a666f9d005e86ce075dce7687dc49b11"}
Sep 16 16:43:11.672959 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.672481    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:43:11.674204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.674180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" event={"ID":"86791c8b-0f46-4f45-a713-7f8dfe9d78fa","Type":"ContainerStarted","Data":"0100c6c024205f4d62fe5f1da14abe1596a5e92d8f0f2be78046c305b9b21fd0"}
Sep 16 16:43:11.690590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.690527    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b55tq" podStartSLOduration=373.723142049 podStartE2EDuration="6m15.690505279s" totalImagesPullingTime="1.96736323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:43:11.689248922 +0000 UTC m=+408.774968995" watchObservedRunningTime="2026-09-16 16:43:11.690505279 +0000 UTC m=+408.776225352"
Sep 16 16:43:11.711634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:11.706911    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-smsn2" podStartSLOduration=382.306479055 podStartE2EDuration="6m23.706890409s" totalImagesPullingTime="1.400411354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:43:11.704093848 +0000 UTC m=+408.789813979" watchObservedRunningTime="2026-09-16 16:43:11.706890409 +0000 UTC m=+408.792610484"
Sep 16 16:43:21.681323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:43:21.681280    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b55tq"
Sep 16 16:44:04.012874 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:04.012832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:44:04.030855 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:04.030823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:44:04.899005 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:04.898972    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 16:44:39.392527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.392488    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"]
Sep 16 16:44:39.396480 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.396445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.403153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.403122    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 16:44:39.403749 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.403726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rcpct\""
Sep 16 16:44:39.403883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.403862    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 16:44:39.416598 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.416555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"]
Sep 16 16:44:39.537543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.537502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vph5\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-kube-api-access-9vph5\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.537725 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.537565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.638161 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.638120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.638318 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.638190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vph5\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-kube-api-access-9vph5\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.647045 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.646963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.647045 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.647033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vph5\" (UniqueName: \"kubernetes.io/projected/d71f8628-7bad-4107-b947-fb9d01dc2cf6-kube-api-access-9vph5\") pod \"cert-manager-cainjector-559cdd8c87-kxnvk\" (UID: \"d71f8628-7bad-4107-b947-fb9d01dc2cf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:39.721201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:39.721165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"
Sep 16 16:44:40.000137 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:40.000038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk"]
Sep 16 16:44:40.003168 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:44:40.003136    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd71f8628_7bad_4107_b947_fb9d01dc2cf6.slice/crio-d30fbe729a9913a1a1be05036acfa479768a661283a109fc22b45b648719fac1 WatchSource:0}: Error finding container d30fbe729a9913a1a1be05036acfa479768a661283a109fc22b45b648719fac1: Status 404 returned error can't find the container with id d30fbe729a9913a1a1be05036acfa479768a661283a109fc22b45b648719fac1
Sep 16 16:44:40.021449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:40.021410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk" event={"ID":"d71f8628-7bad-4107-b947-fb9d01dc2cf6","Type":"ContainerStarted","Data":"d30fbe729a9913a1a1be05036acfa479768a661283a109fc22b45b648719fac1"}
Sep 16 16:44:44.039744 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:44.039706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk" event={"ID":"d71f8628-7bad-4107-b947-fb9d01dc2cf6","Type":"ContainerStarted","Data":"4d62d734b44f8d2bb2003f657618b57ac062f3255a66a7039e3829575a7e222c"}
Sep 16 16:44:44.056098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:44.056034    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxnvk" podStartSLOduration=1.522099158 podStartE2EDuration="5.056014615s" totalImagesPullingTime="3.533915457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:44:44.054822134 +0000 UTC m=+501.140542206" watchObservedRunningTime="2026-09-16 16:44:44.056014615 +0000 UTC m=+501.141734687"
Sep 16 16:44:56.515507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.515469    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sckph"]
Sep 16 16:44:56.522027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.521996    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.524479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.524441    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f4nkz\""
Sep 16 16:44:56.529732 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.529697    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sckph"]
Sep 16 16:44:56.690213 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.690152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxgjg\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-kube-api-access-dxgjg\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.690213 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.690220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-bound-sa-token\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.791259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.791223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-bound-sa-token\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.791374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.791317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxgjg\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-kube-api-access-dxgjg\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.800007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.799970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-bound-sa-token\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.800184 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.800163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxgjg\" (UniqueName: \"kubernetes.io/projected/ecd24922-80d4-4381-b88b-9fa87815dade-kube-api-access-dxgjg\") pod \"cert-manager-55b4d954d5-sckph\" (UID: \"ecd24922-80d4-4381-b88b-9fa87815dade\") " pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:56.832977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:56.832940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sckph"
Sep 16 16:44:57.107791 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:57.107702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sckph"]
Sep 16 16:44:57.110758 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:44:57.110713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecd24922_80d4_4381_b88b_9fa87815dade.slice/crio-0f2020d010508205b187ba16ae772796272a2cb418de24f887d96607944f7dd2 WatchSource:0}: Error finding container 0f2020d010508205b187ba16ae772796272a2cb418de24f887d96607944f7dd2: Status 404 returned error can't find the container with id 0f2020d010508205b187ba16ae772796272a2cb418de24f887d96607944f7dd2
Sep 16 16:44:58.097479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:58.097444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sckph" event={"ID":"ecd24922-80d4-4381-b88b-9fa87815dade","Type":"ContainerStarted","Data":"3dc801d99cc671f92491741fa926faae3fcc7a327e191049df0fab1069c21348"}
Sep 16 16:44:58.097879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:58.097488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sckph" event={"ID":"ecd24922-80d4-4381-b88b-9fa87815dade","Type":"ContainerStarted","Data":"0f2020d010508205b187ba16ae772796272a2cb418de24f887d96607944f7dd2"}
Sep 16 16:44:58.112865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:44:58.112806    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sckph" podStartSLOduration=2.11277495 podStartE2EDuration="2.11277495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:44:58.11156414 +0000 UTC m=+515.197284213" watchObservedRunningTime="2026-09-16 16:44:58.11277495 +0000 UTC m=+515.198495021"
Sep 16 16:45:27.584701 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.584621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"]
Sep 16 16:45:27.587844 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.587819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.590129 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.590101    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 16:45:27.590276 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.590142    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 16:45:27.590455 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.590432    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 16:45:27.590516 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.590453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 16:45:27.590570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.590549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mhjfp\""
Sep 16 16:45:27.610090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.610051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"]
Sep 16 16:45:27.663261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.663221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.663261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.663264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gc48\" (UniqueName: \"kubernetes.io/projected/5ab49632-219f-48f6-9dc2-ae10a6602707-kube-api-access-2gc48\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.663530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.663291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.663530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.663366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.663530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.663465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764373 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764571 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gc48\" (UniqueName: \"kubernetes.io/projected/5ab49632-219f-48f6-9dc2-ae10a6602707-kube-api-access-2gc48\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764571 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764571 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764571 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.764885 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.764861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.766926 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.766888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab49632-219f-48f6-9dc2-ae10a6602707-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.767041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.766951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.767114 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.767098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab49632-219f-48f6-9dc2-ae10a6602707-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.776173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.776144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gc48\" (UniqueName: \"kubernetes.io/projected/5ab49632-219f-48f6-9dc2-ae10a6602707-kube-api-access-2gc48\") pod \"opendatahub-operator-controller-manager-54d7c7f658-k699z\" (UID: \"5ab49632-219f-48f6-9dc2-ae10a6602707\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:27.899553 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:27.899443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:28.190178 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:28.190148    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"]
Sep 16 16:45:28.192734 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:45:28.192705    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab49632_219f_48f6_9dc2_ae10a6602707.slice/crio-d58bc0e2b78535159f9116c9d80d670579cb6493a76da6e5753a11828f37248d WatchSource:0}: Error finding container d58bc0e2b78535159f9116c9d80d670579cb6493a76da6e5753a11828f37248d: Status 404 returned error can't find the container with id d58bc0e2b78535159f9116c9d80d670579cb6493a76da6e5753a11828f37248d
Sep 16 16:45:28.227277 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:28.227238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z" event={"ID":"5ab49632-219f-48f6-9dc2-ae10a6602707","Type":"ContainerStarted","Data":"d58bc0e2b78535159f9116c9d80d670579cb6493a76da6e5753a11828f37248d"}
Sep 16 16:45:31.247665 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:31.247622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z" event={"ID":"5ab49632-219f-48f6-9dc2-ae10a6602707","Type":"ContainerStarted","Data":"ec8f52c055f89b8f35c2414761e53c391cf543e36b37f1e0ee68638b01d235c2"}
Sep 16 16:45:33.258217 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:33.258182    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ab49632-219f-48f6-9dc2-ae10a6602707" containerID="ec8f52c055f89b8f35c2414761e53c391cf543e36b37f1e0ee68638b01d235c2" exitCode=0
Sep 16 16:45:33.258645 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:33.258237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z" event={"ID":"5ab49632-219f-48f6-9dc2-ae10a6602707","Type":"ContainerDied","Data":"ec8f52c055f89b8f35c2414761e53c391cf543e36b37f1e0ee68638b01d235c2"}
Sep 16 16:45:34.264690 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:34.264649    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z" event={"ID":"5ab49632-219f-48f6-9dc2-ae10a6602707","Type":"ContainerStarted","Data":"570d3ccf71179fa207456b269f88233aca5ce9c5cae5cb092be6997d93a5e958"}
Sep 16 16:45:34.265201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:34.264886    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:45:34.288953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:34.288898    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z" podStartSLOduration=2.7494972410000003 podStartE2EDuration="7.288879811s" totalImagesPullingTime="3.420473018s" totalInitContainerRuntime="1.118909552s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:45:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:45:34.287756288 +0000 UTC m=+551.373476362" watchObservedRunningTime="2026-09-16 16:45:34.288879811 +0000 UTC m=+551.374599883"
Sep 16 16:45:45.271057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:45:45.271022    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-k699z"
Sep 16 16:46:06.349928 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.349887    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"]
Sep 16 16:46:06.353398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.353367    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.359544 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.359515    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 16:46:06.360487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.360310    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 16:46:06.360586 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.360576    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 16:46:06.360780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.360761    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gcg2f\""
Sep 16 16:46:06.391207 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.391170    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"]
Sep 16 16:46:06.519479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnzt9\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-kube-api-access-tnzt9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.519897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.519797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.620800 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnzt9\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-kube-api-access-tnzt9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.621147 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.621147 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.620950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.621271 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.621242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.621446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.621422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.621530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.621512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.623327 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.623303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.623515 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.623492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.631461 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.631426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnzt9\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-kube-api-access-tnzt9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.631814 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.631788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb947a5-3b59-40e6-8efe-3ede5e7bd29d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2\" (UID: \"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.665394 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.665338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:06.945614 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:06.945531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"]
Sep 16 16:46:06.949036 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:46:06.949005    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb947a5_3b59_40e6_8efe_3ede5e7bd29d.slice/crio-7f50b622fa54d5bc44610e214f1313759395f6566a9a829e8e47f7a97bcc9c81 WatchSource:0}: Error finding container 7f50b622fa54d5bc44610e214f1313759395f6566a9a829e8e47f7a97bcc9c81: Status 404 returned error can't find the container with id 7f50b622fa54d5bc44610e214f1313759395f6566a9a829e8e47f7a97bcc9c81
Sep 16 16:46:07.410716 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:07.410674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2" event={"ID":"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d","Type":"ContainerStarted","Data":"7f50b622fa54d5bc44610e214f1313759395f6566a9a829e8e47f7a97bcc9c81"}
Sep 16 16:46:08.156453 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.156413    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"]
Sep 16 16:46:08.159977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.159952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.163347 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163313    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 16:46:08.163672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163649    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 16:46:08.163777 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163677    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-52z2s\""
Sep 16 16:46:08.163842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163772    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 16:46:08.163842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163789    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 16:46:08.163944 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.163910    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:46:08.171575 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.171543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"]
Sep 16 16:46:08.237077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.237039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.237273 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.237101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc9vh\" (UniqueName: \"kubernetes.io/projected/4158574b-8d54-47e3-a17b-4a8c69d924b2-kube-api-access-sc9vh\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.237273 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.237203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4158574b-8d54-47e3-a17b-4a8c69d924b2-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.237273 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.237253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.344893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.342827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc9vh\" (UniqueName: \"kubernetes.io/projected/4158574b-8d54-47e3-a17b-4a8c69d924b2-kube-api-access-sc9vh\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.344893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.342893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4158574b-8d54-47e3-a17b-4a8c69d924b2-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.344893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.342930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.344893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.343023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.344893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.344213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4158574b-8d54-47e3-a17b-4a8c69d924b2-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.349747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.349711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.349968 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.349842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4158574b-8d54-47e3-a17b-4a8c69d924b2-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.357319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.357271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc9vh\" (UniqueName: \"kubernetes.io/projected/4158574b-8d54-47e3-a17b-4a8c69d924b2-kube-api-access-sc9vh\") pod \"lws-controller-manager-c5d8fb6c6-j6wkb\" (UID: \"4158574b-8d54-47e3-a17b-4a8c69d924b2\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.474510 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.474416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:08.777346 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:08.777251    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"]
Sep 16 16:46:08.782560 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:46:08.782527    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4158574b_8d54_47e3_a17b_4a8c69d924b2.slice/crio-86e416454700cdff874792fa07af3a373855c256748fa8690d875160d8ef39c0 WatchSource:0}: Error finding container 86e416454700cdff874792fa07af3a373855c256748fa8690d875160d8ef39c0: Status 404 returned error can't find the container with id 86e416454700cdff874792fa07af3a373855c256748fa8690d875160d8ef39c0
Sep 16 16:46:09.392899 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.392862    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"]
Sep 16 16:46:09.398051 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.398022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.402932 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.402668    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 16 16:46:09.402932 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.402709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-h999f\""
Sep 16 16:46:09.417814 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.417784    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"]
Sep 16 16:46:09.421579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.421547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.425090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.424882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-76w7d\""
Sep 16 16:46:09.425873 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.425339    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 16 16:46:09.436561 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.436523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb" event={"ID":"4158574b-8d54-47e3-a17b-4a8c69d924b2","Type":"ContainerStarted","Data":"86e416454700cdff874792fa07af3a373855c256748fa8690d875160d8ef39c0"}
Sep 16 16:46:09.453848 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.453279    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"]
Sep 16 16:46:09.461080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.461043    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"]
Sep 16 16:46:09.555354 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.555311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7v8q\" (UniqueName: \"kubernetes.io/projected/43095e77-16dc-4cec-a9fd-a684496902ed-kube-api-access-q7v8q\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.555863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.555473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3a1c5c9-5019-44bf-8103-3b285fa9893e-cert\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.555863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.555506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43095e77-16dc-4cec-a9fd-a684496902ed-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.555863 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.555625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhkj\" (UniqueName: \"kubernetes.io/projected/f3a1c5c9-5019-44bf-8103-3b285fa9893e-kube-api-access-wbhkj\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.657142 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.657041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhkj\" (UniqueName: \"kubernetes.io/projected/f3a1c5c9-5019-44bf-8103-3b285fa9893e-kube-api-access-wbhkj\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.657326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.657152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7v8q\" (UniqueName: \"kubernetes.io/projected/43095e77-16dc-4cec-a9fd-a684496902ed-kube-api-access-q7v8q\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.657326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.657195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3a1c5c9-5019-44bf-8103-3b285fa9893e-cert\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.657326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.657220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43095e77-16dc-4cec-a9fd-a684496902ed-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.660564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.660427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3a1c5c9-5019-44bf-8103-3b285fa9893e-cert\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.660981 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.660880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43095e77-16dc-4cec-a9fd-a684496902ed-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.669881 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.669845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7v8q\" (UniqueName: \"kubernetes.io/projected/43095e77-16dc-4cec-a9fd-a684496902ed-kube-api-access-q7v8q\") pod \"llmisvc-controller-manager-777c6bd7d5-zs5g5\" (UID: \"43095e77-16dc-4cec-a9fd-a684496902ed\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:09.670554 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.670482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhkj\" (UniqueName: \"kubernetes.io/projected/f3a1c5c9-5019-44bf-8103-3b285fa9893e-kube-api-access-wbhkj\") pod \"kserve-controller-manager-d6f4b65b8-4xkbz\" (UID: \"f3a1c5c9-5019-44bf-8103-3b285fa9893e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.712816 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.712769    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:09.737319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:09.737282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:10.066636 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:10.066600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"]
Sep 16 16:46:10.088767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:10.088659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"]
Sep 16 16:46:11.128290 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:46:11.128244    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a1c5c9_5019_44bf_8103_3b285fa9893e.slice/crio-6781d8733cb01069c7971fb9ac25e8424e4c26128c3e7fcccf3a7a30ae55af41 WatchSource:0}: Error finding container 6781d8733cb01069c7971fb9ac25e8424e4c26128c3e7fcccf3a7a30ae55af41: Status 404 returned error can't find the container with id 6781d8733cb01069c7971fb9ac25e8424e4c26128c3e7fcccf3a7a30ae55af41
Sep 16 16:46:11.448648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:11.448559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz" event={"ID":"f3a1c5c9-5019-44bf-8103-3b285fa9893e","Type":"ContainerStarted","Data":"6781d8733cb01069c7971fb9ac25e8424e4c26128c3e7fcccf3a7a30ae55af41"}
Sep 16 16:46:11.858818 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:46:11.858780    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod43095e77_16dc_4cec_a9fd_a684496902ed.slice/crio-1647fbb382eb6b1eb3e2f89b444494d653493d43f9c33bf6592e7daded0b45c4 WatchSource:0}: Error finding container 1647fbb382eb6b1eb3e2f89b444494d653493d43f9c33bf6592e7daded0b45c4: Status 404 returned error can't find the container with id 1647fbb382eb6b1eb3e2f89b444494d653493d43f9c33bf6592e7daded0b45c4
Sep 16 16:46:12.297229 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.297170    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:46:12.297672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.297287    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:46:12.297672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.297332    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:46:12.457891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.457785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb" event={"ID":"4158574b-8d54-47e3-a17b-4a8c69d924b2","Type":"ContainerStarted","Data":"78c99be999c7d929cc094b26a0ccee433e15e8559c19aece71495cb4765d931c"}
Sep 16 16:46:12.458093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.457990    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:12.460918 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.460127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2" event={"ID":"dbb947a5-3b59-40e6-8efe-3ede5e7bd29d","Type":"ContainerStarted","Data":"4e94d9834efe9aa7a79369ddb2db0611fe9698f624646654724459e6a73762d9"}
Sep 16 16:46:12.464024 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.463832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5" event={"ID":"43095e77-16dc-4cec-a9fd-a684496902ed","Type":"ContainerStarted","Data":"1647fbb382eb6b1eb3e2f89b444494d653493d43f9c33bf6592e7daded0b45c4"}
Sep 16 16:46:12.478229 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.478126    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb" podStartSLOduration=0.911825092 podStartE2EDuration="4.478102659s" totalImagesPullingTime="3.566277567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:12.47795021 +0000 UTC m=+589.563670283" watchObservedRunningTime="2026-09-16 16:46:12.478102659 +0000 UTC m=+589.563822724"
Sep 16 16:46:12.529728 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.529647    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2" podStartSLOduration=1.183869759 podStartE2EDuration="6.529621483s" totalImagesPullingTime="5.345751724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:12.529510526 +0000 UTC m=+589.615230599" watchObservedRunningTime="2026-09-16 16:46:12.529621483 +0000 UTC m=+589.615341556"
Sep 16 16:46:12.666016 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:12.665917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:13.672937 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:13.672891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:14.476134 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:14.476096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:14.477354 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:14.477329    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2"
Sep 16 16:46:15.482607 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.482569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz" event={"ID":"f3a1c5c9-5019-44bf-8103-3b285fa9893e","Type":"ContainerStarted","Data":"78f60f6669c549738ddd4eb38259bb8f6341397717ef45563c72a2e5aac5e16a"}
Sep 16 16:46:15.483096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.482690    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:46:15.484237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.484208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5" event={"ID":"43095e77-16dc-4cec-a9fd-a684496902ed","Type":"ContainerStarted","Data":"05c053412f30a92d789b58271f725863842f71d61be7c83de4623f2f59ef09ac"}
Sep 16 16:46:15.484367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.484339    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:15.501203 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.501141    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz" podStartSLOduration=2.331462085 podStartE2EDuration="6.50112423s" totalImagesPullingTime="4.169662145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:15.499134859 +0000 UTC m=+592.584854932" watchObservedRunningTime="2026-09-16 16:46:15.50112423 +0000 UTC m=+592.586844298"
Sep 16 16:46:15.523269 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:15.523211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5" podStartSLOduration=3.083315638 podStartE2EDuration="6.523193123s" totalImagesPullingTime="3.439877485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:15.520480592 +0000 UTC m=+592.606200675" watchObservedRunningTime="2026-09-16 16:46:15.523193123 +0000 UTC m=+592.608913195"
Sep 16 16:46:23.472990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:23.472955    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-j6wkb"
Sep 16 16:46:46.491930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:46.491840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zs5g5"
Sep 16 16:46:46.502895 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:46:46.495084    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4xkbz"
Sep 16 16:47:19.003671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.003625    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"]
Sep 16 16:47:19.006090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.006056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:19.008987 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.008959    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 16 16:47:19.009146 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.008959    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cnlxr\""
Sep 16 16:47:19.009976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.009950    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 16:47:19.010099 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.009977    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 16:47:19.023362 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.023332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"]
Sep 16 16:47:19.123723 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.123678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpb9\" (UniqueName: \"kubernetes.io/projected/579f7a0c-5ff4-44cd-874e-fac14378a226-kube-api-access-hmpb9\") pod \"dns-operator-controller-manager-7b84cd7c9c-z6t9j\" (UID: \"579f7a0c-5ff4-44cd-874e-fac14378a226\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:19.224912 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.224868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpb9\" (UniqueName: \"kubernetes.io/projected/579f7a0c-5ff4-44cd-874e-fac14378a226-kube-api-access-hmpb9\") pod \"dns-operator-controller-manager-7b84cd7c9c-z6t9j\" (UID: \"579f7a0c-5ff4-44cd-874e-fac14378a226\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:19.238840 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.238809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpb9\" (UniqueName: \"kubernetes.io/projected/579f7a0c-5ff4-44cd-874e-fac14378a226-kube-api-access-hmpb9\") pod \"dns-operator-controller-manager-7b84cd7c9c-z6t9j\" (UID: \"579f7a0c-5ff4-44cd-874e-fac14378a226\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:19.322751 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.322711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:19.606360 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.606259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"]
Sep 16 16:47:19.609956 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:47:19.609917    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579f7a0c_5ff4_44cd_874e_fac14378a226.slice/crio-d02539f12e1d242822624d553f91f22641db8c8819cee625791ce625abce2f7c WatchSource:0}: Error finding container d02539f12e1d242822624d553f91f22641db8c8819cee625791ce625abce2f7c: Status 404 returned error can't find the container with id d02539f12e1d242822624d553f91f22641db8c8819cee625791ce625abce2f7c
Sep 16 16:47:19.612319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.612298    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:47:19.789201 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:19.789159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j" event={"ID":"579f7a0c-5ff4-44cd-874e-fac14378a226","Type":"ContainerStarted","Data":"d02539f12e1d242822624d553f91f22641db8c8819cee625791ce625abce2f7c"}
Sep 16 16:47:22.810833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:22.810800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j" event={"ID":"579f7a0c-5ff4-44cd-874e-fac14378a226","Type":"ContainerStarted","Data":"a849d17cbf78b44e5530fdecc53cd192740cdc5baabeeeddbad44310380668a4"}
Sep 16 16:47:22.811231 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:22.810876    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:22.829172 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:22.829111    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j" podStartSLOduration=2.308130588 podStartE2EDuration="4.829094135s" totalImagesPullingTime="2.520963547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:47:22.828364942 +0000 UTC m=+659.914085018" watchObservedRunningTime="2026-09-16 16:47:22.829094135 +0000 UTC m=+659.914814206"
Sep 16 16:47:25.934499 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:25.934461    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gvzmp"]
Sep 16 16:47:25.937520 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:25.937492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:25.941494 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:25.941459    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-z4cql\""
Sep 16 16:47:25.953697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:25.953661    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gvzmp"]
Sep 16 16:47:26.097319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.097282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ww9\" (UniqueName: \"kubernetes.io/projected/56db9976-26c8-40da-9745-d08b0b8b3df8-kube-api-access-84ww9\") pod \"authorino-operator-76b478bd78-gvzmp\" (UID: \"56db9976-26c8-40da-9745-d08b0b8b3df8\") " pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:26.198742 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.198653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84ww9\" (UniqueName: \"kubernetes.io/projected/56db9976-26c8-40da-9745-d08b0b8b3df8-kube-api-access-84ww9\") pod \"authorino-operator-76b478bd78-gvzmp\" (UID: \"56db9976-26c8-40da-9745-d08b0b8b3df8\") " pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:26.216240 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.216201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ww9\" (UniqueName: \"kubernetes.io/projected/56db9976-26c8-40da-9745-d08b0b8b3df8-kube-api-access-84ww9\") pod \"authorino-operator-76b478bd78-gvzmp\" (UID: \"56db9976-26c8-40da-9745-d08b0b8b3df8\") " pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:26.249969 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.249925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:26.540746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.540717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gvzmp"]
Sep 16 16:47:26.542820 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:47:26.542785    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56db9976_26c8_40da_9745_d08b0b8b3df8.slice/crio-813af31804092b326210cebccef8d8c5e46bf8bcfd246bdd77c790c4d1c0b768 WatchSource:0}: Error finding container 813af31804092b326210cebccef8d8c5e46bf8bcfd246bdd77c790c4d1c0b768: Status 404 returned error can't find the container with id 813af31804092b326210cebccef8d8c5e46bf8bcfd246bdd77c790c4d1c0b768
Sep 16 16:47:26.831982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:26.831885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp" event={"ID":"56db9976-26c8-40da-9745-d08b0b8b3df8","Type":"ContainerStarted","Data":"813af31804092b326210cebccef8d8c5e46bf8bcfd246bdd77c790c4d1c0b768"}
Sep 16 16:47:28.843930 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:28.843887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp" event={"ID":"56db9976-26c8-40da-9745-d08b0b8b3df8","Type":"ContainerStarted","Data":"ca4d6b729486ed051af12ec151b8345e8803ee027210c023a2ee103328a8ff01"}
Sep 16 16:47:28.844370 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:28.844021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:47:28.861088 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:28.861034    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp" podStartSLOduration=2.389729481 podStartE2EDuration="3.861015813s" totalImagesPullingTime="1.471286332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:47:28.860463173 +0000 UTC m=+665.946183247" watchObservedRunningTime="2026-09-16 16:47:28.861015813 +0000 UTC m=+665.946735885"
Sep 16 16:47:33.818272 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:33.818239    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-z6t9j"
Sep 16 16:47:39.852024 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:47:39.851991    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-gvzmp"
Sep 16 16:48:10.668705 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.668658    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"]
Sep 16 16:48:10.675084 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.675056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.677605 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.677575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5v52j\""
Sep 16 16:48:10.685251 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.685215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"]
Sep 16 16:48:10.799537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e5a7999-9a09-4657-bc86-47fa7733c92f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq7zc\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-kube-api-access-bq7zc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.799815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.799812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.900771 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.900771 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7zc\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-kube-api-access-bq7zc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.900932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901357 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.901340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901439 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.901407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901496 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.901437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e5a7999-9a09-4657-bc86-47fa7733c92f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901596 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.901575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.901699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.901682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.902145 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.902121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e5a7999-9a09-4657-bc86-47fa7733c92f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.903417 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.903377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.903702 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.903683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.909923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.909884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.910107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.910079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq7zc\" (UniqueName: \"kubernetes.io/projected/8e5a7999-9a09-4657-bc86-47fa7733c92f-kube-api-access-bq7zc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vwq2p\" (UID: \"8e5a7999-9a09-4657-bc86-47fa7733c92f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:10.987568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:10.987468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:11.270474 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:11.270440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"]
Sep 16 16:48:11.273117 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:48:11.273086    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5a7999_9a09_4657_bc86_47fa7733c92f.slice/crio-4f709ac9d2730fb05895dcc61682a67e71165f05fcb3e18bcf2cabfcff434dfb WatchSource:0}: Error finding container 4f709ac9d2730fb05895dcc61682a67e71165f05fcb3e18bcf2cabfcff434dfb: Status 404 returned error can't find the container with id 4f709ac9d2730fb05895dcc61682a67e71165f05fcb3e18bcf2cabfcff434dfb
Sep 16 16:48:11.275540 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:11.275499    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:48:11.275631 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:11.275580    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:48:11.275631 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:11.275607    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:48:12.054249 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:12.054197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" event={"ID":"8e5a7999-9a09-4657-bc86-47fa7733c92f","Type":"ContainerStarted","Data":"54260cd6abbffb36f248538bbfec6e53f5471a08a4294c51b2c347e2b9ec3d2e"}
Sep 16 16:48:12.054756 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:12.054321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" event={"ID":"8e5a7999-9a09-4657-bc86-47fa7733c92f","Type":"ContainerStarted","Data":"4f709ac9d2730fb05895dcc61682a67e71165f05fcb3e18bcf2cabfcff434dfb"}
Sep 16 16:48:12.078781 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:12.078715    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podStartSLOduration=2.078699234 podStartE2EDuration="2.078699234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:48:12.076684382 +0000 UTC m=+709.162404454" watchObservedRunningTime="2026-09-16 16:48:12.078699234 +0000 UTC m=+709.164419306"
Sep 16 16:48:12.988036 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:12.987994    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:12.993404 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:12.993359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:13.059609 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:13.059576    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:13.060826 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:13.060802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:48:27.427188 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.427099    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:48:27.430771 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.430745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.433069 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.433043    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kkrlh\""
Sep 16 16:48:27.433195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.433043    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 16:48:27.438996 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.438968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:48:27.467262 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.467223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:48:27.549756 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.549719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-config-file\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.550001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.549784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brk27\" (UniqueName: \"kubernetes.io/projected/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-kube-api-access-brk27\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.650567 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.650527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-config-file\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.650745 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.650598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brk27\" (UniqueName: \"kubernetes.io/projected/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-kube-api-access-brk27\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.651259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.651239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-config-file\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.659853 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.659818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brk27\" (UniqueName: \"kubernetes.io/projected/3ff645b4-76c6-48ab-84be-319f7b8ba8e0-kube-api-access-brk27\") pod \"limitador-limitador-84994f7ffc-88ttm\" (UID: \"3ff645b4-76c6-48ab-84be-319f7b8ba8e0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:27.742320 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:27.742221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:28.023433 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:28.021025    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:48:28.028446 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:48:28.028406    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff645b4_76c6_48ab_84be_319f7b8ba8e0.slice/crio-8787b67f918d55730b582579183d223932d3759cb5ec21b0065b142c54ab5b87 WatchSource:0}: Error finding container 8787b67f918d55730b582579183d223932d3759cb5ec21b0065b142c54ab5b87: Status 404 returned error can't find the container with id 8787b67f918d55730b582579183d223932d3759cb5ec21b0065b142c54ab5b87
Sep 16 16:48:28.133867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:28.133827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm" event={"ID":"3ff645b4-76c6-48ab-84be-319f7b8ba8e0","Type":"ContainerStarted","Data":"8787b67f918d55730b582579183d223932d3759cb5ec21b0065b142c54ab5b87"}
Sep 16 16:48:30.146243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:30.146197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm" event={"ID":"3ff645b4-76c6-48ab-84be-319f7b8ba8e0","Type":"ContainerStarted","Data":"68d66625e6f34add1a4d578805ba558e39cee9e55307f56ce90f84da22fe01af"}
Sep 16 16:48:30.146693 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:30.146352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:48:30.165206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:30.165152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm" podStartSLOduration=2.112961207 podStartE2EDuration="3.165134267s" totalImagesPullingTime="1.05217306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:48:30.163448385 +0000 UTC m=+727.249168457" watchObservedRunningTime="2026-09-16 16:48:30.165134267 +0000 UTC m=+727.250854339"
Sep 16 16:48:41.152091 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:48:41.152060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-88ttm"
Sep 16 16:49:41.173515 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:41.173473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:49:50.542317 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.542285    2577 generic.go:356] "Generic (PLEG): container finished" podID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerID="54260cd6abbffb36f248538bbfec6e53f5471a08a4294c51b2c347e2b9ec3d2e" exitCode=0
Sep 16 16:49:50.542760 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.542370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" event={"ID":"8e5a7999-9a09-4657-bc86-47fa7733c92f","Type":"ContainerDied","Data":"54260cd6abbffb36f248538bbfec6e53f5471a08a4294c51b2c347e2b9ec3d2e"}
Sep 16 16:49:50.542832 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.542817    2577 scope.go:117] "RemoveContainer" containerID="54260cd6abbffb36f248538bbfec6e53f5471a08a4294c51b2c347e2b9ec3d2e"
Sep 16 16:49:50.543897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.543866    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:49:50.543973 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.543943    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:49:50.544012 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:50.543987    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:49:51.552072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:51.552036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" event={"ID":"8e5a7999-9a09-4657-bc86-47fa7733c92f","Type":"ContainerStarted","Data":"74f5c01d7aa67ac7f329c31ba5dd5f32a62f9e6b0668b51a399dd70a6e2179cc"}
Sep 16 16:49:51.988485 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:51.988357    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:49:51.988641 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:51.988594    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vwq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 16 16:49:51.988683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:51.988642    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podUID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 16 16:49:52.988887 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:52.988844    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vwq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 16 16:49:52.989301 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:52.988965    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podUID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 16 16:49:53.988339 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:53.988291    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vwq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 16 16:49:53.988568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:53.988359    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podUID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 16 16:49:54.988726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:54.988682    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vwq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 16 16:49:54.989203 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:54.988744    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podUID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 16 16:49:55.988500 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:55.988449    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vwq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 16 16:49:55.988722 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:55.988528    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p" podUID="8e5a7999-9a09-4657-bc86-47fa7733c92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 16 16:49:56.102002 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:56.101962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:49:57.193355 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:57.193320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:49:57.193826 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:57.193746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:49:57.194649 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:49:57.194628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vwq2p"
Sep 16 16:50:09.583098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.583051    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"]
Sep 16 16:50:09.585826 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.585803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.588945 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.588899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cxtbk\""
Sep 16 16:50:09.589116 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.588899    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 16:50:09.589116 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.588983    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 16:50:09.589116 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.588997    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 16 16:50:09.596082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.596051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"]
Sep 16 16:50:09.689118 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.689118 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.689351 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c5c92ff-5e97-4859-995f-b7f57ee97940-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.689351 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzmcj\" (UniqueName: \"kubernetes.io/projected/5c5c92ff-5e97-4859-995f-b7f57ee97940-kube-api-access-bzmcj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.689351 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.689351 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.689294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790448 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c5c92ff-5e97-4859-995f-b7f57ee97940-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790448 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzmcj\" (UniqueName: \"kubernetes.io/projected/5c5c92ff-5e97-4859-995f-b7f57ee97940-kube-api-access-bzmcj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790448 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790448 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790723 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.790833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.790772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.792806 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.792779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c5c92ff-5e97-4859-995f-b7f57ee97940-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.793114 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.793087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c5c92ff-5e97-4859-995f-b7f57ee97940-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.799502 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.799473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzmcj\" (UniqueName: \"kubernetes.io/projected/5c5c92ff-5e97-4859-995f-b7f57ee97940-kube-api-access-bzmcj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-czj4z\" (UID: \"5c5c92ff-5e97-4859-995f-b7f57ee97940\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:09.898424 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:09.898327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:10.181503 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:10.181413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"]
Sep 16 16:50:10.184859 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:50:10.184828    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5c92ff_5e97_4859_995f_b7f57ee97940.slice/crio-fd98a79d01f1ada2299a3e636f47591a4caa281a8bfac82a13b593716fb76b69 WatchSource:0}: Error finding container fd98a79d01f1ada2299a3e636f47591a4caa281a8bfac82a13b593716fb76b69: Status 404 returned error can't find the container with id fd98a79d01f1ada2299a3e636f47591a4caa281a8bfac82a13b593716fb76b69
Sep 16 16:50:10.372077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:10.372044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:50:10.657003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:10.656962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z" event={"ID":"5c5c92ff-5e97-4859-995f-b7f57ee97940","Type":"ContainerStarted","Data":"fd98a79d01f1ada2299a3e636f47591a4caa281a8bfac82a13b593716fb76b69"}
Sep 16 16:50:11.664672 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:11.664571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z" event={"ID":"5c5c92ff-5e97-4859-995f-b7f57ee97940","Type":"ContainerStarted","Data":"cb2ac1b5afb8b86d553c7990103a754024b83946fbb76ac8084cc69c14aaf8c7"}
Sep 16 16:50:11.681531 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:11.681471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z" podStartSLOduration=1.537432841 podStartE2EDuration="2.681450249s" totalImagesPullingTime="1.144017408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:50:11.680475413 +0000 UTC m=+828.766195486" watchObservedRunningTime="2026-09-16 16:50:11.681450249 +0000 UTC m=+828.767170321"
Sep 16 16:50:19.898935 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:19.898899    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:19.899449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:19.898970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:19.911856 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:19.911824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:20.720606 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:20.720571    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-czj4z"
Sep 16 16:50:26.411454 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:26.411412    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:50:32.901998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:32.901947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:50:43.700317 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:43.700280    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:50:48.418016 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:48.417974    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:50:57.083032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.082991    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"]
Sep 16 16:50:57.086811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.086785    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.089181 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.089153    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 16:50:57.097077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.097048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"]
Sep 16 16:50:57.134751 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.134947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.134947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.134947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.134947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbqt6\" (UniqueName: \"kubernetes.io/projected/e2a270cf-6aa7-434e-9c3a-0e9436eda014-kube-api-access-rbqt6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.134947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.134903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.235984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.235942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.235984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbqt6\" (UniqueName: \"kubernetes.io/projected/e2a270cf-6aa7-434e-9c3a-0e9436eda014-kube-api-access-rbqt6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236516 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.236579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.236542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.238557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.238534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2a270cf-6aa7-434e-9c3a-0e9436eda014-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.238850 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.238827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2a270cf-6aa7-434e-9c3a-0e9436eda014-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.246065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.246028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbqt6\" (UniqueName: \"kubernetes.io/projected/e2a270cf-6aa7-434e-9c3a-0e9436eda014-kube-api-access-rbqt6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd\" (UID: \"e2a270cf-6aa7-434e-9c3a-0e9436eda014\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.397878 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.397779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:50:57.673227 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.673173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"]
Sep 16 16:50:57.675588 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:50:57.675556    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2a270cf_6aa7_434e_9c3a_0e9436eda014.slice/crio-45e69d3ac9d220de526b014fd8b04f0f781753e776965bfbe87c47d2e4a3552e WatchSource:0}: Error finding container 45e69d3ac9d220de526b014fd8b04f0f781753e776965bfbe87c47d2e4a3552e: Status 404 returned error can't find the container with id 45e69d3ac9d220de526b014fd8b04f0f781753e776965bfbe87c47d2e4a3552e
Sep 16 16:50:57.902735 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:57.902701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd" event={"ID":"e2a270cf-6aa7-434e-9c3a-0e9436eda014","Type":"ContainerStarted","Data":"45e69d3ac9d220de526b014fd8b04f0f781753e776965bfbe87c47d2e4a3552e"}
Sep 16 16:50:58.909735 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:58.909696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd" event={"ID":"e2a270cf-6aa7-434e-9c3a-0e9436eda014","Type":"ContainerStarted","Data":"bc39c2a226205dd37de4e85f88d3d11e9898486bb34c23395d454a0bfa3ebc0d"}
Sep 16 16:50:58.934612 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:58.934511    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd" podStartSLOduration=1.758281652 podStartE2EDuration="1.934478806s" totalImagesPullingTime="176.197154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:50:58.927120882 +0000 UTC m=+876.012840982" watchObservedRunningTime="2026-09-16 16:50:58.934478806 +0000 UTC m=+876.020198883"
Sep 16 16:50:59.608693 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:50:59.608652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:51:07.398703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:51:07.398661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:51:07.399183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:51:07.398972    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:51:07.411679 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:51:07.411647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:51:07.969401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:51:07.969349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd"
Sep 16 16:52:03.549368 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.549327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:52:03.553291 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.553267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.555649 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.555625    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-eb2ab2-openshift-default-dockercfg-mz66s\""
Sep 16 16:52:03.565743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.565903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.565903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.565903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.566027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.566027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.565978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.566027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.566016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.566122 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.566042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7gjw\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.566122 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.566090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.573995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.573962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:52:03.667678 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.667639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.667938 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.667905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7gjw\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668695 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668695 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.668984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.668953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.669401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.669366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.671267 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.671240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.671984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.671957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.693969 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.693934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.695837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.695808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7gjw\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw\") pod \"e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") " pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.747371 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.747329    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:52:03.752157 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.752124    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.755891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.755864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e2ea3e-openshift-default-dockercfg-dlf6j\""
Sep 16 16:52:03.769567 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769567 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv2dz\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.769828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.770086 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.770086 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.769885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.785323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.785287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:52:03.866138 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.866047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:03.870694 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.870659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.870810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.870719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.870810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.870792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.870921 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.870866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.870975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.870900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.871032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.871015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.871082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.871056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.871138 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.871097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.871183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.871161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv2dz\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.872047 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.872020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.872245 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.872189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.872329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.872264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.872488 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.872461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.872718 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.872693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.874187 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.874139    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.875104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.875083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.887565 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.887458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:03.889026 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:03.888959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv2dz\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz\") pod \"e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") " pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:04.064654 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.064607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:04.162649 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.162612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:52:04.165286 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:04.165255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03015ef9_3088_43b8_b16f_35d12421533a.slice/crio-39cf981b35d2ca795607a1379712152c6f0e0e7ce3d9fd3b5985c82151af2a5c WatchSource:0}: Error finding container 39cf981b35d2ca795607a1379712152c6f0e0e7ce3d9fd3b5985c82151af2a5c: Status 404 returned error can't find the container with id 39cf981b35d2ca795607a1379712152c6f0e0e7ce3d9fd3b5985c82151af2a5c
Sep 16 16:52:04.167879 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.167840    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.168000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.167927    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.168000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.167970    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.262648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.262606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerStarted","Data":"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"}
Sep 16 16:52:04.262848 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.262674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerStarted","Data":"39cf981b35d2ca795607a1379712152c6f0e0e7ce3d9fd3b5985c82151af2a5c"}
Sep 16 16:52:04.306692 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.306637    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podStartSLOduration=1.306617316 podStartE2EDuration="1.306617316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:04.304135316 +0000 UTC m=+941.389855389" watchObservedRunningTime="2026-09-16 16:52:04.306617316 +0000 UTC m=+941.392337423"
Sep 16 16:52:04.379147 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.379052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:52:04.381751 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:04.381720    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04acd5e2_add6_45d4_885a_db96de8daed9.slice/crio-f58035efd8722fc3b9a66dc91baf18608df6a1cea9fc3e26ed0e1d375ad10e92 WatchSource:0}: Error finding container f58035efd8722fc3b9a66dc91baf18608df6a1cea9fc3e26ed0e1d375ad10e92: Status 404 returned error can't find the container with id f58035efd8722fc3b9a66dc91baf18608df6a1cea9fc3e26ed0e1d375ad10e92
Sep 16 16:52:04.384834 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.384788    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.384977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.384884    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.384977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.384928    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:04.866162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:04.866127    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:05.269776 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:05.269676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerStarted","Data":"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"}
Sep 16 16:52:05.269776 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:05.269744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerStarted","Data":"f58035efd8722fc3b9a66dc91baf18608df6a1cea9fc3e26ed0e1d375ad10e92"}
Sep 16 16:52:05.296927 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:05.296868    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podStartSLOduration=2.296851201 podStartE2EDuration="2.296851201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:05.293424199 +0000 UTC m=+942.379144273" watchObservedRunningTime="2026-09-16 16:52:05.296851201 +0000 UTC m=+942.382571272"
Sep 16 16:52:05.323682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:05.323642    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:52:05.871922 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:05.871892    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:06.065226 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.065189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:06.070376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.070346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:06.275978 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.275876    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:06.276155 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.275979    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:06.276711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.276688    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:06.276842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:06.276740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:11.600840 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.600801    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:52:11.604902 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.604869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.607240 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.607208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-708c6a-openshift-default-dockercfg-2d8hs\""
Sep 16 16:52:11.620158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.619648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:52:11.652586 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652794 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652794 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652794 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652931 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6prwn\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652974 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.652974 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.653077 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.652995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.653134 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.653089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.754711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6prwn\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.755058 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.755058 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.754981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.755157 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.755126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.755309 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.755287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.755590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.755569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.757090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.757067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.757651 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.757621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.762452 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.762420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6prwn\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.762699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.762675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token\") pod \"e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") " pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:11.919747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:11.919650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:12.204350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.204177    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:52:12.210465 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:12.210429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6990ec_3c4f_4a43_9690_6d443d870e0a.slice/crio-0f05cd2cbef6480019643c68a1ed5b439b2cf8aeb5b79b282cd33a2a9e938bb9 WatchSource:0}: Error finding container 0f05cd2cbef6480019643c68a1ed5b439b2cf8aeb5b79b282cd33a2a9e938bb9: Status 404 returned error can't find the container with id 0f05cd2cbef6480019643c68a1ed5b439b2cf8aeb5b79b282cd33a2a9e938bb9
Sep 16 16:52:12.212839 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.212799    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:12.212975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.212910    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:12.212975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.212961    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:12.312911 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.312871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerStarted","Data":"0fddc0dfa1ca5670771450002b01d1f4ec47b08e7e79ceeeea20becfdf8607dd"}
Sep 16 16:52:12.313098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.312939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerStarted","Data":"0f05cd2cbef6480019643c68a1ed5b439b2cf8aeb5b79b282cd33a2a9e938bb9"}
Sep 16 16:52:12.335624 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.335568    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podStartSLOduration=1.3355498319999999 podStartE2EDuration="1.335549832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:12.333129252 +0000 UTC m=+949.418849336" watchObservedRunningTime="2026-09-16 16:52:12.335549832 +0000 UTC m=+949.421269904"
Sep 16 16:52:12.663008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.662969    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:52:12.666867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.666842    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.669058 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.669032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-41cc4972-openshift-default-dockercfg-79nm6\""
Sep 16 16:52:12.681033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.680995    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:52:12.683866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.683836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:52:12.765904 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.765867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.765929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.765957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.765988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.766018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.766114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.766224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.766263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.766447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.766311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8tvp\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.867862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.867824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.867862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.867866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8tvp\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.867935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.867962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.867978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.868951 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.868932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.869081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.869038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.870640 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.870618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.870749 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.870720    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.878513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.878475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.878641 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.878628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8tvp\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp\") pod \"e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") " pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:12.920291 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.920202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:12.980209 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:12.980159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:13.294719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.294690    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:52:13.296801 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:13.296769    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a89584_6aef_4610_89f6_fea98165c7be.slice/crio-5cd4b76fece69f95f058582f383f621259ea48f56527e5e14615d3ebff3f69e4 WatchSource:0}: Error finding container 5cd4b76fece69f95f058582f383f621259ea48f56527e5e14615d3ebff3f69e4: Status 404 returned error can't find the container with id 5cd4b76fece69f95f058582f383f621259ea48f56527e5e14615d3ebff3f69e4
Sep 16 16:52:13.299376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.299337    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:13.299485 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.299431    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:13.299485 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.299466    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:13.325455 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.325345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerStarted","Data":"5cd4b76fece69f95f058582f383f621259ea48f56527e5e14615d3ebff3f69e4"}
Sep 16 16:52:13.945410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:13.938751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:14.336995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.336948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerStarted","Data":"4f9781585cf006010a8365c895d782d4efd1859237d92100ad21bae69a3394eb"}
Sep 16 16:52:14.337710 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.337679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:14.346018 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.345974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:14.357468 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.357369    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" podStartSLOduration=2.357345006 podStartE2EDuration="2.357345006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:14.356354783 +0000 UTC m=+951.442074855" watchObservedRunningTime="2026-09-16 16:52:14.357345006 +0000 UTC m=+951.443065081"
Sep 16 16:52:14.980547 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.980462    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:14.997289 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:14.997256    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:15.347796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:15.347759    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:15.358512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:15.358478    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:20.047180 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.047136    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:20.051811 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.051784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.058240    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.058564    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.058785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.058994    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.059148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-41cc4972-serving-cert\""
Sep 16 16:52:20.060057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.059292    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:20.160781 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.160722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.160985 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.160829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.160985 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.160858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.161096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.161009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d44qv\" (UniqueName: \"kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.262603 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.262505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.262764 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.262607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.262764 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.262637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.262764 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.262715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d44qv\" (UniqueName: \"kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.262942 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:20.262780    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-serving-cert: secret "maas-api-e2e-ait-41cc4972-serving-cert" not found
Sep 16 16:52:20.262942 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:20.262867    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls podName:71f389fe-6ce5-474b-b4f1-83cb4c6de71f nodeName:}" failed. No retries permitted until 2026-09-16 16:52:20.762845655 +0000 UTC m=+957.848565721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls") pod "maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f") : secret "maas-api-e2e-ait-41cc4972-serving-cert" not found
Sep 16 16:52:20.263488 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.263464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.265804 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.265768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.277148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.277119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d44qv\" (UniqueName: \"kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.768429 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:20.768374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:20.768615 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:20.768576    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-serving-cert: secret "maas-api-e2e-ait-41cc4972-serving-cert" not found
Sep 16 16:52:20.768683 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:20.768666    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls podName:71f389fe-6ce5-474b-b4f1-83cb4c6de71f nodeName:}" failed. No retries permitted until 2026-09-16 16:52:21.768639717 +0000 UTC m=+958.854359788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls") pod "maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f") : secret "maas-api-e2e-ait-41cc4972-serving-cert" not found
Sep 16 16:52:21.782223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:21.782170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:21.785070 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:21.785033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:21.871032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:21.870984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:22.180088 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.180060    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:22.182618 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:22.182571    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f389fe_6ce5_474b_b4f1_83cb4c6de71f.slice/crio-8003cb7a16729d4e2d4baece085073a97620174675bd429641571f39bd4e1608 WatchSource:0}: Error finding container 8003cb7a16729d4e2d4baece085073a97620174675bd429641571f39bd4e1608: Status 404 returned error can't find the container with id 8003cb7a16729d4e2d4baece085073a97620174675bd429641571f39bd4e1608
Sep 16 16:52:22.184816 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.184790    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:52:22.318607 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.318563    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 16:52:22.318779 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.318640    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 16:52:22.318779 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.318562    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 16:52:22.318779 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.318724    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 16:52:22.401396 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.401351    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" event={"ID":"71f389fe-6ce5-474b-b4f1-83cb4c6de71f","Type":"ContainerStarted","Data":"8003cb7a16729d4e2d4baece085073a97620174675bd429641571f39bd4e1608"}
Sep 16 16:52:22.747973 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.747927    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:52:22.767966 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.767932    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.771342 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.771315    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ea8f09-openshift-default-dockercfg-d4vh7\""
Sep 16 16:52:22.809231 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.809190    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.895981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9xn\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.896557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.896347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.998192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.998192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9xn\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:22.999432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:22.998470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.001462 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.001103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.001462 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.001350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.002175 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.001943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.002175 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.002122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.002802 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.002756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.003088 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.003067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.005564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.005516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.015353 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.015289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.016723 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.016663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9xn\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn\") pod \"e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") " pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.083508 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.083425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:23.394723 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:23.394682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6157255_be2b_4e13_aa49_8ee1eb8d54a7.slice/crio-4dd688dbbea702795945f17a599c455d7cde04e60bff948183d257531e02b0d0 WatchSource:0}: Error finding container 4dd688dbbea702795945f17a599c455d7cde04e60bff948183d257531e02b0d0: Status 404 returned error can't find the container with id 4dd688dbbea702795945f17a599c455d7cde04e60bff948183d257531e02b0d0
Sep 16 16:52:23.398328 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.397483    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.398328 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.397593    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.398328 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.397640    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.398681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.398651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:52:23.413990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.413939    2577 generic.go:356] "Generic (PLEG): container finished" podID="03015ef9-3088-43b8-b16f-35d12421533a" containerID="657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87" exitCode=0
Sep 16 16:52:23.414192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.414027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerDied","Data":"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"}
Sep 16 16:52:23.414646 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.414621    2577 scope.go:117] "RemoveContainer" containerID="657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"
Sep 16 16:52:23.417864 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.417284    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.417864 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.417413    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.417864 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.417470    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:23.422706 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:23.418511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerStarted","Data":"4dd688dbbea702795945f17a599c455d7cde04e60bff948183d257531e02b0d0"}
Sep 16 16:52:24.462125 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.461135    2577 generic.go:356] "Generic (PLEG): container finished" podID="04acd5e2-add6-45d4-885a-db96de8daed9" containerID="e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890" exitCode=0
Sep 16 16:52:24.462720 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.461243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerDied","Data":"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"}
Sep 16 16:52:24.462996 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.462808    2577 scope.go:117] "RemoveContainer" containerID="e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"
Sep 16 16:52:24.466329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.466290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerStarted","Data":"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"}
Sep 16 16:52:24.468356 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.468330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerStarted","Data":"4dc305946256d617dd70b3c52d802a86f3160a60b27e45ac997b77e7981ff2c4"}
Sep 16 16:52:24.550938 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.550867    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podStartSLOduration=2.550838059 podStartE2EDuration="2.550838059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:24.542966627 +0000 UTC m=+961.628686701" watchObservedRunningTime="2026-09-16 16:52:24.550838059 +0000 UTC m=+961.636558131"
Sep 16 16:52:24.758467 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.758341    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:24.758611 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.758476    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:24.758611 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.758512    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:24.866374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.866324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:24.866630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.866607    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:24.866700 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:24.866657    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:25.084052 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.083924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:25.089912 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.089877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:25.475211 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.474851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:25.475870 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.475848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:25.866455 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.866416    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:25.866644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:25.866475    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:26.483265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:26.483225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerStarted","Data":"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"}
Sep 16 16:52:26.484881 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:26.484849    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" event={"ID":"71f389fe-6ce5-474b-b4f1-83cb4c6de71f","Type":"ContainerStarted","Data":"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"}
Sep 16 16:52:26.485017 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:26.485009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:26.866562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:26.866509    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:26.866802 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:26.866587    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:27.065265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:27.065221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:27.065699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:27.065665    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Sep 16 16:52:27.065816 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:27.065747    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Sep 16 16:52:27.866884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:27.866833    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:27.867450 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:27.866910    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:28.065243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:28.065198    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Sep 16 16:52:28.065575 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:28.065536    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Sep 16 16:52:28.867331 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:28.867286    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:28.868000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:28.867962    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:29.068331 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:29.067856    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Sep 16 16:52:29.068331 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:29.067923    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Sep 16 16:52:29.867692 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:29.867624    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 16 16:52:29.868240 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:29.867737    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 16 16:52:30.065314 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.065266    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Sep 16 16:52:30.065526 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.065329    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Sep 16 16:52:30.871013 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.870979    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:30.871430 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.871278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:30.872011 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.871993    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:52:30.893904 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:30.893845    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" podStartSLOduration=7.662344161 podStartE2EDuration="10.893826729s" totalImagesPullingTime="3.231482568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:26.551727877 +0000 UTC m=+963.637447950" watchObservedRunningTime="2026-09-16 16:52:30.893826729 +0000 UTC m=+967.979546800"
Sep 16 16:52:31.065628 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:31.065585    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Sep 16 16:52:31.065817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:31.065643    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Sep 16 16:52:32.068933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:32.068895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:32.069482 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:32.069303    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:32.069953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:32.069934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:52:32.497940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:32.497855    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:33.118630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:33.118595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:52:45.378616 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:45.378565    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 16:52:45.406129 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:45.378639    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 16:52:46.322734 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.304605    2577 patch_prober.go:28] interesting pod/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:34774->10.132.0.37:15021: read: connection reset by peer" start-of-body=
Sep 16 16:52:46.322734 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.304689    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:34774->10.132.0.37:15021: read: connection reset by peer"
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.632042    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5a89584-6aef-4610-89f6-fea98165c7be" containerID="4f9781585cf006010a8365c895d782d4efd1859237d92100ad21bae69a3394eb" exitCode=0
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.632116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerDied","Data":"4f9781585cf006010a8365c895d782d4efd1859237d92100ad21bae69a3394eb"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.632690    2577 scope.go:117] "RemoveContainer" containerID="4f9781585cf006010a8365c895d782d4efd1859237d92100ad21bae69a3394eb"
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634003    2577 generic.go:356] "Generic (PLEG): container finished" podID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerID="0fddc0dfa1ca5670771450002b01d1f4ec47b08e7e79ceeeea20becfdf8607dd" exitCode=0
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerDied","Data":"0fddc0dfa1ca5670771450002b01d1f4ec47b08e7e79ceeeea20becfdf8607dd"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634198    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634268    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634302    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.634457    2577 scope.go:117] "RemoveContainer" containerID="0fddc0dfa1ca5670771450002b01d1f4ec47b08e7e79ceeeea20becfdf8607dd"
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.635514    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.635575    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:46.715420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:46.635607    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:47.650419 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.645976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerStarted","Data":"6e23325a897fb40cea94f2bba9367c9a8462e13f95e52e9d6413b9fdf5bf6c04"}
Sep 16 16:52:47.655401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.653989    2577 generic.go:356] "Generic (PLEG): container finished" podID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerID="4dc305946256d617dd70b3c52d802a86f3160a60b27e45ac997b77e7981ff2c4" exitCode=0
Sep 16 16:52:47.655401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.654114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerDied","Data":"4dc305946256d617dd70b3c52d802a86f3160a60b27e45ac997b77e7981ff2c4"}
Sep 16 16:52:47.655401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.654688    2577 scope.go:117] "RemoveContainer" containerID="4dc305946256d617dd70b3c52d802a86f3160a60b27e45ac997b77e7981ff2c4"
Sep 16 16:52:47.658006 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.657961    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:47.658731 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.658702    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:47.658923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.658903    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:47.670574 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.670538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerStarted","Data":"6e39d343053edb87dcd1a562bea847d29b0024c103154d79c7b1f7886056d20b"}
Sep 16 16:52:47.921101 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.920938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:47.921101 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.920989    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:47.921101 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.921047    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:47.980977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.980935    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:47.981432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.981198    2577 patch_prober.go:28] interesting pod/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 16 16:52:47.981432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:47.981275    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 16 16:52:48.046030 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.045979    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:48.046277 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.046245    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" podUID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" containerName="maas-api" containerID="cri-o://03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769" gracePeriod=30
Sep 16 16:52:48.571408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.567609    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:48.585730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.585692    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:52:48.688408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.687245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d44qv\" (UniqueName: \"kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv\") pod \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") "
Sep 16 16:52:48.688408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.687313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca\") pod \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") "
Sep 16 16:52:48.688408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.687505    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs\") pod \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") "
Sep 16 16:52:48.688408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.687546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") pod \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\" (UID: \"71f389fe-6ce5-474b-b4f1-83cb4c6de71f\") "
Sep 16 16:52:48.696407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.693274    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71f389fe-6ce5-474b-b4f1-83cb4c6de71f" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:52:48.710545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.705949    2577 generic.go:356] "Generic (PLEG): container finished" podID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" containerID="03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769" exitCode=0
Sep 16 16:52:48.710545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.706152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" event={"ID":"71f389fe-6ce5-474b-b4f1-83cb4c6de71f","Type":"ContainerDied","Data":"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"}
Sep 16 16:52:48.710545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.706212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh" event={"ID":"71f389fe-6ce5-474b-b4f1-83cb4c6de71f","Type":"ContainerDied","Data":"8003cb7a16729d4e2d4baece085073a97620174675bd429641571f39bd4e1608"}
Sep 16 16:52:48.710545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.706244    2577 scope.go:117] "RemoveContainer" containerID="03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"
Sep 16 16:52:48.710545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.706433    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"
Sep 16 16:52:48.793410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.789677    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.803410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.802010    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv" (OuterVolumeSpecName: "kube-api-access-d44qv") pod "71f389fe-6ce5-474b-b4f1-83cb4c6de71f" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f"). InnerVolumeSpecName "kube-api-access-d44qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:48.803410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.802496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "71f389fe-6ce5-474b-b4f1-83cb4c6de71f" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:52:48.803410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.802870    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71f389fe-6ce5-474b-b4f1-83cb4c6de71f" (UID: "71f389fe-6ce5-474b-b4f1-83cb4c6de71f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:52:48.846412 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.834297    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" secret="" err="secret \"e2e-ait-41cc4972-openshift-default-dockercfg-79nm6\" not found"
Sep 16 16:52:48.846412 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.834916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerStarted","Data":"b37f1071da1cb3b38df8f54f98eedac2bf9cd5fd561720e2d945ce93deb07f44"}
Sep 16 16:52:48.855605 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.855573    2577 scope.go:117] "RemoveContainer" containerID="03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"
Sep 16 16:52:48.864421 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:48.863924    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769\": container with ID starting with 03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769 not found: ID does not exist" containerID="03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"
Sep 16 16:52:48.864421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.863978    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769"} err="failed to get container status \"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769\": rpc error: code = NotFound desc = could not find container \"03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769\": container with ID starting with 03de64492f63a5d58a4587d5a2d199b2942fa73620e9348c232be15a90207769 not found: ID does not exist"
Sep 16 16:52:48.895409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.892252    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d44qv\" (UniqueName: \"kubernetes.io/projected/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-kube-api-access-d44qv\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.895409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.892296    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.895409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.892319    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71f389fe-6ce5-474b-b4f1-83cb4c6de71f-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.921168 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.921123    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:48.921860 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.921823    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:48.982073 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.981603    2577 patch_prober.go:28] interesting pod/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 16 16:52:48.982073 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:48.981668    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 16 16:52:49.068205 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.068107    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:49.084302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.084249    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:49.084911 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.084491    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:49.084911 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.084646    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:49.089587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.089511    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-41cc4972-6474c588f4-dj2wh"]
Sep 16 16:52:49.398141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.398035    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" path="/var/lib/kubelet/pods/71f389fe-6ce5-474b-b4f1-83cb4c6de71f/volumes"
Sep 16 16:52:49.541407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.540580    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:52:49.541407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.541242    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" containerName="maas-api"
Sep 16 16:52:49.541407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.541265    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" containerName="maas-api"
Sep 16 16:52:49.541407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.541405    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f389fe-6ce5-474b-b4f1-83cb4c6de71f" containerName="maas-api"
Sep 16 16:52:49.554444 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.551953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.558410 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.557748    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-241a48eb-gw-openshift-default-dockercfg-slbpw\""
Sep 16 16:52:49.568414 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.562275    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.600910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.600990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrm8p\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.603413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.601304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.702938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrm8p\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.703487 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.703334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.707407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.704200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.707407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.705150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.707407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.705777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.707407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.706011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.707407 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.706225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.732449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.726341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.732449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.726697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.732449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.731102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.732931 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.732904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrm8p\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p\") pod \"e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") " pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.845538 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.844442    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy" containerID="cri-o://6e39d343053edb87dcd1a562bea847d29b0024c103154d79c7b1f7886056d20b" gracePeriod=30
Sep 16 16:52:49.883681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.883646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:49.921281 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.920811    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:49.921281 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:49.920878    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:50.085063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.084577    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:50.085063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.084647    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:50.370529 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.370423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:52:50.394633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.394215    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:50.394633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.394323    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:50.394633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.394370    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:52:50.882468 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.882432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerStarted","Data":"8aeb21e07df622966d823408ac03da0c499602c2bd7a8b3c5c14fb841d2c8885"}
Sep 16 16:52:50.882832 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.882809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerStarted","Data":"114e1a040235b0b8be062ddc60b778d77198f7c3ae4c4893bf5b566060e1885c"}
Sep 16 16:52:50.888971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.888895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:50.911408 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.908377    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podStartSLOduration=1.9083556659999998 podStartE2EDuration="1.908355666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:50.90751188 +0000 UTC m=+987.993231953" watchObservedRunningTime="2026-09-16 16:52:50.908355666 +0000 UTC m=+987.994075739"
Sep 16 16:52:50.921623 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.920112    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:50.921623 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:50.920175    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:51.084221 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.084182    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:51.084543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.084505    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:51.894175 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.894140    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:52:51.894509 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.894476    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:52:51.921207 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.921158    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:51.921538 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:51.921488    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:52.084301 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.084265    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:52.084979 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.084928    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:52.102505 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.102236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:52:52.898471 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.898123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:52.901240 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.901206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:52.902324 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.902301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:52:52.921136 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.921097    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 16 16:52:52.921343 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:52.921179    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 16 16:52:53.090411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.085872    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:53.090411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.085941    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:53.928115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.928079    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:53.929972 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.929942    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:53.931265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.931238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:52:53.958287 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.958252    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:52:53.964350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.964317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:53.966677 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.966648    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-241a48eb-serving-cert\""
Sep 16 16:52:53.966677 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.966655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:52:53.966876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.966694    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:52:53.966876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.966700    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:52:53.966876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.966700    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:52:53.970838 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:53.970771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:52:54.084979 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.084596    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 16 16:52:54.084979 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.084657    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 16 16:52:54.164001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.163364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.164001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.163481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mlh\" (UniqueName: \"kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.164001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.163562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.164001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.163618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.210438 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.210321    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:52:54.215227 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.215199    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.217822 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.217790    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-eb2ab2\"/\"openshift-service-ca.crt\""
Sep 16 16:52:54.217976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.217928    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-eb2ab2\"/\"default-dockercfg-rlxs4\""
Sep 16 16:52:54.217976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.217943    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-eb2ab2\"/\"kube-root-ca.crt\""
Sep 16 16:52:54.217976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.217952    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-eb2ab2\"/\"simulator-w0-eb2ab2-kserve-self-signed-certs\""
Sep 16 16:52:54.229725 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.229685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:52:54.264537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.264733 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.264733 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.264733 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:54.264689    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-serving-cert: secret "maas-api-e2e-migrate-241a48eb-serving-cert" not found
Sep 16 16:52:54.264940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.264940 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:52:54.264767    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls podName:625b3ab4-1d22-4655-b85d-f87932a4d1da nodeName:}" failed. No retries permitted until 2026-09-16 16:52:54.764738402 +0000 UTC m=+991.850458466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls") pod "maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" (UID: "625b3ab4-1d22-4655-b85d-f87932a4d1da") : secret "maas-api-e2e-migrate-241a48eb-serving-cert" not found
Sep 16 16:52:54.264940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jp6v\" (UniqueName: \"kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.264940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mlh\" (UniqueName: \"kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.265158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.265158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.264970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.265158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.265015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.265158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.265053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.265793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.265773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.267833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.267804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.273681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.273641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mlh\" (UniqueName: \"kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.366033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.365989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366200 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366200 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366446 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jp6v\" (UniqueName: \"kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366515 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366576 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.366630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.366580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.368788 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.368742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.369066 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.369043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.379156 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.379121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jp6v\" (UniqueName: \"kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v\") pod \"simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") " pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.533865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.533760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:52:54.778075 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.775101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.782406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.782174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") pod \"maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.824696 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.824668    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:52:54.827579 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:54.827551    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod745834cc_3e2a_4984_8ceb_50a48068c0ec.slice/crio-cc924dba82f433f4bb28abb3d2b22f933f7c0453ac96d72f0d10d3ee45755cee WatchSource:0}: Error finding container cc924dba82f433f4bb28abb3d2b22f933f7c0453ac96d72f0d10d3ee45755cee: Status 404 returned error can't find the container with id cc924dba82f433f4bb28abb3d2b22f933f7c0453ac96d72f0d10d3ee45755cee
Sep 16 16:52:54.882308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.882274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:54.916271 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:54.916235    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" event={"ID":"745834cc-3e2a-4984-8ceb-50a48068c0ec","Type":"ContainerStarted","Data":"cc924dba82f433f4bb28abb3d2b22f933f7c0453ac96d72f0d10d3ee45755cee"}
Sep 16 16:52:55.088451 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.088327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:55.088692 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.088668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:55.089620 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.089600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:52:55.171190 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.170744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:52:55.173846 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:52:55.173818    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625b3ab4_1d22_4655_b85d_f87932a4d1da.slice/crio-1e7661b1d7d334df7c0a0745ad2c673971a4f3431e67b511d5b6ef2ab2dfa3b9 WatchSource:0}: Error finding container 1e7661b1d7d334df7c0a0745ad2c673971a4f3431e67b511d5b6ef2ab2dfa3b9: Status 404 returned error can't find the container with id 1e7661b1d7d334df7c0a0745ad2c673971a4f3431e67b511d5b6ef2ab2dfa3b9
Sep 16 16:52:55.924776 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.924720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" event={"ID":"745834cc-3e2a-4984-8ceb-50a48068c0ec","Type":"ContainerStarted","Data":"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"}
Sep 16 16:52:55.926590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.926553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" event={"ID":"625b3ab4-1d22-4655-b85d-f87932a4d1da","Type":"ContainerStarted","Data":"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"}
Sep 16 16:52:55.926729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.926622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:52:55.926729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.926643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" event={"ID":"625b3ab4-1d22-4655-b85d-f87932a4d1da","Type":"ContainerStarted","Data":"1e7661b1d7d334df7c0a0745ad2c673971a4f3431e67b511d5b6ef2ab2dfa3b9"}
Sep 16 16:52:55.943014 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:55.942956    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" podStartSLOduration=1.942936824 podStartE2EDuration="1.942936824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:55.940803818 +0000 UTC m=+993.026523892" watchObservedRunningTime="2026-09-16 16:52:55.942936824 +0000 UTC m=+993.028656896"
Sep 16 16:52:58.952296 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:58.952203    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5a89584-6aef-4610-89f6-fea98165c7be" containerID="6e39d343053edb87dcd1a562bea847d29b0024c103154d79c7b1f7886056d20b" exitCode=0
Sep 16 16:52:58.952296 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:58.952248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerDied","Data":"6e39d343053edb87dcd1a562bea847d29b0024c103154d79c7b1f7886056d20b"}
Sep 16 16:52:58.952766 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:58.952321    2577 scope.go:117] "RemoveContainer" containerID="4f9781585cf006010a8365c895d782d4efd1859237d92100ad21bae69a3394eb"
Sep 16 16:52:59.212679 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.212605    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:59.233798 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.233737    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" podStartSLOduration=5.91292512 podStartE2EDuration="6.233721181s" totalImagesPullingTime="320.796061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:55.95846513 +0000 UTC m=+993.044185202" watchObservedRunningTime="2026-09-16 16:52:59.233721181 +0000 UTC m=+996.319441253"
Sep 16 16:52:59.326123 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326086    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326123 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8tvp\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326402 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326680 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:52:59.326680 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326440    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data\") pod \"b5a89584-6aef-4610-89f6-fea98165c7be\" (UID: \"b5a89584-6aef-4610-89f6-fea98165c7be\") "
Sep 16 16:52:59.326829 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:52:59.326970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:52:59.326970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326929    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:52:59.326970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.326956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:52:59.327139 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.327031    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.327139 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.327055    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.328779 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.328738    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:52:59.328883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.328825    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:59.328883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.328853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:52:59.328883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.328868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp" (OuterVolumeSpecName: "kube-api-access-d8tvp") pod "b5a89584-6aef-4610-89f6-fea98165c7be" (UID: "b5a89584-6aef-4610-89f6-fea98165c7be"). InnerVolumeSpecName "kube-api-access-d8tvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:59.428406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428340    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428375    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a89584-6aef-4610-89f6-fea98165c7be-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428399    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428410    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a89584-6aef-4610-89f6-fea98165c7be-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428422    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a89584-6aef-4610-89f6-fea98165c7be-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428434    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8tvp\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-kube-api-access-d8tvp\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.428683 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.428444    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a89584-6aef-4610-89f6-fea98165c7be-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:52:59.960041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.960001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x" event={"ID":"b5a89584-6aef-4610-89f6-fea98165c7be","Type":"ContainerDied","Data":"5cd4b76fece69f95f058582f383f621259ea48f56527e5e14615d3ebff3f69e4"}
Sep 16 16:52:59.960041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.960022    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"
Sep 16 16:52:59.960550 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.960061    2577 scope.go:117] "RemoveContainer" containerID="6e39d343053edb87dcd1a562bea847d29b0024c103154d79c7b1f7886056d20b"
Sep 16 16:52:59.978228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.978193    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:52:59.984877 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:52:59.984842    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-41cc4972-openshift-default-57d98d5c8-55m4x"]
Sep 16 16:53:01.391409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:01.391355    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" path="/var/lib/kubelet/pods/b5a89584-6aef-4610-89f6-fea98165c7be/volumes"
Sep 16 16:53:01.938034 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:01.938004    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:53:04.534897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:04.534859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:53:04.535502 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:04.534915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:53:04.558941 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:04.558910    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:53:05.076223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:05.076194    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:53:11.514140 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:11.514091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:53:12.037158 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.037121    2577 generic.go:356] "Generic (PLEG): container finished" podID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerID="8aeb21e07df622966d823408ac03da0c499602c2bd7a8b3c5c14fb841d2c8885" exitCode=0
Sep 16 16:53:12.037400 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.037193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerDied","Data":"8aeb21e07df622966d823408ac03da0c499602c2bd7a8b3c5c14fb841d2c8885"}
Sep 16 16:53:12.037818 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.037802    2577 scope.go:117] "RemoveContainer" containerID="8aeb21e07df622966d823408ac03da0c499602c2bd7a8b3c5c14fb841d2c8885"
Sep 16 16:53:12.039048 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.039020    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:12.039112 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.039093    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:12.039152 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:12.039135    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:13.049187 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:13.049143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerStarted","Data":"b7b5db6d372cbf88a8c665dd265dafc8509908684c1104239671cd326cb5badd"}
Sep 16 16:53:13.889278 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:13.889227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:13.889561 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:13.889537    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:53:13.889652 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:13.889597    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:53:14.889961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:14.889923    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:53:14.890364 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:14.889985    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:53:15.890141 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:15.890080    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:53:15.890684 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:15.890169    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:53:16.889367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:16.889324    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:53:16.889585 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:16.889406    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:53:17.889127 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:17.889081    2577 patch_prober.go:28] interesting pod/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 16 16:53:17.889675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:17.889154    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 16 16:53:18.893527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:18.893494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:18.894033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:18.893800    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:18.894726 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:18.894705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:24.558699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.558652    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:53:24.559205 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.558934    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" podUID="625b3ab4-1d22-4655-b85d-f87932a4d1da" containerName="maas-api" containerID="cri-o://97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c" gracePeriod=30
Sep 16 16:53:24.828498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.828456    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:53:24.911061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911020    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9mlh\" (UniqueName: \"kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh\") pod \"625b3ab4-1d22-4655-b85d-f87932a4d1da\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") "
Sep 16 16:53:24.911061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs\") pod \"625b3ab4-1d22-4655-b85d-f87932a4d1da\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") "
Sep 16 16:53:24.911378 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca\") pod \"625b3ab4-1d22-4655-b85d-f87932a4d1da\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") "
Sep 16 16:53:24.911378 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") pod \"625b3ab4-1d22-4655-b85d-f87932a4d1da\" (UID: \"625b3ab4-1d22-4655-b85d-f87932a4d1da\") "
Sep 16 16:53:24.911564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911530    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "625b3ab4-1d22-4655-b85d-f87932a4d1da" (UID: "625b3ab4-1d22-4655-b85d-f87932a4d1da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:53:24.911761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.911736    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625b3ab4-1d22-4655-b85d-f87932a4d1da-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:24.913982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.913951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh" (OuterVolumeSpecName: "kube-api-access-f9mlh") pod "625b3ab4-1d22-4655-b85d-f87932a4d1da" (UID: "625b3ab4-1d22-4655-b85d-f87932a4d1da"). InnerVolumeSpecName "kube-api-access-f9mlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:53:24.914346 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.914315    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "625b3ab4-1d22-4655-b85d-f87932a4d1da" (UID: "625b3ab4-1d22-4655-b85d-f87932a4d1da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:53:24.914474 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:24.914363    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "625b3ab4-1d22-4655-b85d-f87932a4d1da" (UID: "625b3ab4-1d22-4655-b85d-f87932a4d1da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:53:25.012314 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.012275    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9mlh\" (UniqueName: \"kubernetes.io/projected/625b3ab4-1d22-4655-b85d-f87932a4d1da-kube-api-access-f9mlh\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.012314 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.012312    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.012314 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.012324    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625b3ab4-1d22-4655-b85d-f87932a4d1da-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.053752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.053714    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:53:25.053995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.053967    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy" containerID="cri-o://b7b5db6d372cbf88a8c665dd265dafc8509908684c1104239671cd326cb5badd" gracePeriod=30
Sep 16 16:53:25.145363 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.145273    2577 generic.go:356] "Generic (PLEG): container finished" podID="625b3ab4-1d22-4655-b85d-f87932a4d1da" containerID="97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c" exitCode=0
Sep 16 16:53:25.145363 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.145314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" event={"ID":"625b3ab4-1d22-4655-b85d-f87932a4d1da","Type":"ContainerDied","Data":"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"}
Sep 16 16:53:25.145590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.145363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"
Sep 16 16:53:25.145590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.145444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64" event={"ID":"625b3ab4-1d22-4655-b85d-f87932a4d1da","Type":"ContainerDied","Data":"1e7661b1d7d334df7c0a0745ad2c673971a4f3431e67b511d5b6ef2ab2dfa3b9"}
Sep 16 16:53:25.145590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.145480    2577 scope.go:117] "RemoveContainer" containerID="97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"
Sep 16 16:53:25.165939 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.165575    2577 scope.go:117] "RemoveContainer" containerID="97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"
Sep 16 16:53:25.168330 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:53:25.168089    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c\": container with ID starting with 97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c not found: ID does not exist" containerID="97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"
Sep 16 16:53:25.168869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.168735    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c"} err="failed to get container status \"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c\": rpc error: code = NotFound desc = could not find container \"97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c\": container with ID starting with 97478f9cdd83fb6d3c8f68197959df349303a0bd9b362d9de1b3d52c18abd34c not found: ID does not exist"
Sep 16 16:53:25.193729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.193687    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:53:25.196447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.196422    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-241a48eb-5cd546fdc7-7mt64"]
Sep 16 16:53:25.391458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.391419    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625b3ab4-1d22-4655-b85d-f87932a4d1da" path="/var/lib/kubelet/pods/625b3ab4-1d22-4655-b85d-f87932a4d1da/volumes"
Sep 16 16:53:25.856870 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.856822    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:53:25.857350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857303    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625b3ab4-1d22-4655-b85d-f87932a4d1da" containerName="maas-api"
Sep 16 16:53:25.857350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857317    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="625b3ab4-1d22-4655-b85d-f87932a4d1da" containerName="maas-api"
Sep 16 16:53:25.857350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857335    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:25.857350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857341    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:25.857604 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857438    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:25.857604 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.857450    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="625b3ab4-1d22-4655-b85d-f87932a4d1da" containerName="maas-api"
Sep 16 16:53:25.862060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.862031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.865009 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.864986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1fd647e0-openshift-default-dockercfg-fg6pc\""
Sep 16 16:53:25.883164 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.883127    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:53:25.921891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.921848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.921908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.921933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fk2r\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.921985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922093 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.922017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.922130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.922154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.922176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:25.922247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:25.922239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023543 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.023912 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024004 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.024018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024121 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.023900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024193 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.024178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024193 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.024181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024305 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.024230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fk2r\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.024532 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.024505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.026072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.026046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.026578 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.026558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.041993 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.041961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.042217 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.042191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fk2r\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r\") pod \"e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") " pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.174423 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.174297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:26.461494 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.461458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:53:26.464310 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:53:26.464266    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309d8cac_a037_4b83_a3f8_9dbc9c25856a.slice/crio-2dfbd0fa09f86c29096acb54602d77f0db9804c2dbb8b8be536022aa4da19add WatchSource:0}: Error finding container 2dfbd0fa09f86c29096acb54602d77f0db9804c2dbb8b8be536022aa4da19add: Status 404 returned error can't find the container with id 2dfbd0fa09f86c29096acb54602d77f0db9804c2dbb8b8be536022aa4da19add
Sep 16 16:53:26.466888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.466848    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:26.467006 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.466927    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:26.467006 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:26.466958    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:27.160010 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:27.159962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerStarted","Data":"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"}
Sep 16 16:53:27.160466 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:27.160028    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerStarted","Data":"2dfbd0fa09f86c29096acb54602d77f0db9804c2dbb8b8be536022aa4da19add"}
Sep 16 16:53:27.174828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:27.174786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:27.180696 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:27.180665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:27.187369 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:27.187313    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podStartSLOduration=2.187294533 podStartE2EDuration="2.187294533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:27.185737332 +0000 UTC m=+1024.271457404" watchObservedRunningTime="2026-09-16 16:53:27.187294533 +0000 UTC m=+1024.273014632"
Sep 16 16:53:28.172629 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:28.169644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:28.173896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:28.173866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:30.182350 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.182312    2577 generic.go:356] "Generic (PLEG): container finished" podID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerID="b7b5db6d372cbf88a8c665dd265dafc8509908684c1104239671cd326cb5badd" exitCode=0
Sep 16 16:53:30.182854 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.182410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerDied","Data":"b7b5db6d372cbf88a8c665dd265dafc8509908684c1104239671cd326cb5badd"}
Sep 16 16:53:30.182854 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.182479    2577 scope.go:117] "RemoveContainer" containerID="8aeb21e07df622966d823408ac03da0c499602c2bd7a8b3c5c14fb841d2c8885"
Sep 16 16:53:30.314092 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.314059    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:30.372897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.372799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.372897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.372861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.372897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.372885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.372921    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.372998    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrm8p\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373045    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373088    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373116    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:53:30.373210 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373169    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:53:30.373626 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy\") pod \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\" (UID: \"ca11d873-83ef-4bdf-94ac-5ac7281d25b6\") "
Sep 16 16:53:30.373626 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:53:30.373626 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:53:30.373626 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:53:30.373841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373759    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.373841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373774    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.373841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373783    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.373841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373797    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.373841 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.373810    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.375432 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.375403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:53:30.375830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.375792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:53:30.375949 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.375857    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:53:30.376016 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.375967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p" (OuterVolumeSpecName: "kube-api-access-lrm8p") pod "ca11d873-83ef-4bdf-94ac-5ac7281d25b6" (UID: "ca11d873-83ef-4bdf-94ac-5ac7281d25b6"). InnerVolumeSpecName "kube-api-access-lrm8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:53:30.474815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.474771    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.474815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.474807    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.474815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.474816    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:30.474815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:30.474825    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrm8p\" (UniqueName: \"kubernetes.io/projected/ca11d873-83ef-4bdf-94ac-5ac7281d25b6-kube-api-access-lrm8p\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:53:31.160193 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.160154    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:53:31.189570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.189536    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq" event={"ID":"ca11d873-83ef-4bdf-94ac-5ac7281d25b6","Type":"ContainerDied","Data":"114e1a040235b0b8be062ddc60b778d77198f7c3ae4c4893bf5b566060e1885c"}
Sep 16 16:53:31.189570 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.189551    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"
Sep 16 16:53:31.190088 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.189593    2577 scope.go:117] "RemoveContainer" containerID="b7b5db6d372cbf88a8c665dd265dafc8509908684c1104239671cd326cb5badd"
Sep 16 16:53:31.218176 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.218137    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:53:31.224156 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.224116    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-241a48eb-gw-openshift-default-6894f587dd-pmwmq"]
Sep 16 16:53:31.391735 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.391700    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" path="/var/lib/kubelet/pods/ca11d873-83ef-4bdf-94ac-5ac7281d25b6/volumes"
Sep 16 16:53:31.775495 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.775456    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:53:31.776132 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776109    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:31.776132 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776133    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776169    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776179    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776195    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776205    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776305    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a89584-6aef-4610-89f6-fea98165c7be" containerName="istio-proxy"
Sep 16 16:53:31.776323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776320    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.776585 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.776336    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca11d873-83ef-4bdf-94ac-5ac7281d25b6" containerName="istio-proxy"
Sep 16 16:53:31.781237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.781209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.786129 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.786102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:53:31.786493 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.786476    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:53:31.786827 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.786719    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:53:31.786827 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.786731    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1fd647e0-serving-cert\""
Sep 16 16:53:31.787062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.787046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:53:31.797284 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.797253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:53:31.890313 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.890265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.890566 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.890327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfw4\" (UniqueName: \"kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.890657 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.890572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.890657 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.890633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.991491 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.991433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.991491 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.991487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.991767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.991505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.991767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.991552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfw4\" (UniqueName: \"kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.991767 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:53:31.991721    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-serving-cert: secret "maas-api-e2e-ait-1fd647e0-serving-cert" not found
Sep 16 16:53:31.991924 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:53:31.991811    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls podName:b73aac24-d043-48c0-9d84-2ecd4cfdfe6d nodeName:}" failed. No retries permitted until 2026-09-16 16:53:32.491790106 +0000 UTC m=+1029.577510171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls") pod "maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" (UID: "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d") : secret "maas-api-e2e-ait-1fd647e0-serving-cert" not found
Sep 16 16:53:31.992113 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.992091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:31.994243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:31.994213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:32.007065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.007026    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfw4\" (UniqueName: \"kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:32.499413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.499333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:32.501835 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.501808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") pod \"maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:32.588706 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.588666    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:53:32.592620 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.592598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.595084 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.595046    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f604b4\"/\"kube-root-ca.crt\""
Sep 16 16:53:32.595237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.595113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f604b4\"/\"default-dockercfg-c2kzh\""
Sep 16 16:53:32.595701 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.595684    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f604b4\"/\"simulator-w4-f604b4-kserve-self-signed-certs\""
Sep 16 16:53:32.595797 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.595716    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f604b4\"/\"openshift-service-ca.crt\""
Sep 16 16:53:32.607129 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.607084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:53:32.693113 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.693075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:32.701738 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.701705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.701874 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.701768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.701932 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.701905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.701968 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.701947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.702063 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.702047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.702100 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.702071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqd72\" (UniqueName: \"kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.814918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.815009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.815099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.815124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.815210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.815239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqd72\" (UniqueName: \"kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.816082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.816446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.826413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.819584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.830150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.830101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.831830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.831765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.832694 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.832642    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqd72\" (UniqueName: \"kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72\") pod \"simulator-w4-f604b4-kserve-b78c6d699-r2qz9\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") " pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.903458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.903419    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:32.986295 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:32.986258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:53:32.988013 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:53:32.987986    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73aac24_d043_48c0_9d84_2ecd4cfdfe6d.slice/crio-8a0a8eed25765fca252b3bdab550bfc6b83a6d65197332445d89b0120b4a9164 WatchSource:0}: Error finding container 8a0a8eed25765fca252b3bdab550bfc6b83a6d65197332445d89b0120b4a9164: Status 404 returned error can't find the container with id 8a0a8eed25765fca252b3bdab550bfc6b83a6d65197332445d89b0120b4a9164
Sep 16 16:53:33.212071 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:33.212046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:53:33.213165 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:33.213132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" event={"ID":"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d","Type":"ContainerStarted","Data":"8a0a8eed25765fca252b3bdab550bfc6b83a6d65197332445d89b0120b4a9164"}
Sep 16 16:53:33.214431 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:53:33.214401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod047ca784_4c2e_4ab3_80f9_44cbca0c75c2.slice/crio-41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41 WatchSource:0}: Error finding container 41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41: Status 404 returned error can't find the container with id 41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41
Sep 16 16:53:34.221329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.221289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" event={"ID":"047ca784-4c2e-4ab3-80f9-44cbca0c75c2","Type":"ContainerStarted","Data":"e5f530be8de060a616b772700346a70542b37f6ed7eacdf54be3e38f87671f1c"}
Sep 16 16:53:34.221880 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.221340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" event={"ID":"047ca784-4c2e-4ab3-80f9-44cbca0c75c2","Type":"ContainerStarted","Data":"41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41"}
Sep 16 16:53:34.223284 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.223249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" event={"ID":"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d","Type":"ContainerStarted","Data":"67708ae70af047d4f5d6678a4521f5da9934f11bcaec35006cbb92c603edebff"}
Sep 16 16:53:34.223495 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.223331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:34.250401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.250329    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" podStartSLOduration=2.250311408 podStartE2EDuration="2.250311408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:34.247019588 +0000 UTC m=+1031.332739660" watchObservedRunningTime="2026-09-16 16:53:34.250311408 +0000 UTC m=+1031.336031480"
Sep 16 16:53:34.270113 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:34.270043    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" podStartSLOduration=2.8568151139999998 podStartE2EDuration="3.270021287s" totalImagesPullingTime="413.206173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:34.268596365 +0000 UTC m=+1031.354316437" watchObservedRunningTime="2026-09-16 16:53:34.270021287 +0000 UTC m=+1031.355741359"
Sep 16 16:53:40.236269 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:40.236237    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:53:42.903810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:42.903763    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:42.904260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:42.903830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:42.916706 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:42.916667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:43.285937 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:43.285841    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:53:51.205522 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.205475    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:53:51.325534 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.325499    2577 generic.go:356] "Generic (PLEG): container finished" podID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerID="39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2" exitCode=0
Sep 16 16:53:51.325721 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.325577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerDied","Data":"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"}
Sep 16 16:53:51.326128 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.326105    2577 scope.go:117] "RemoveContainer" containerID="39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"
Sep 16 16:53:51.327564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.327526    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:51.327703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.327625    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:51.327703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:51.327678    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:53:52.339041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:52.339005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerStarted","Data":"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"}
Sep 16 16:53:53.174744 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:53.174698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:53.174934 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:53.174911    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 16 16:53:53.175013 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:53.174993    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 16 16:53:54.175083 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:54.175045    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 16 16:53:54.175629 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:54.175107    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 16 16:53:55.174882 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:55.174836    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 16 16:53:55.175104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:55.174910    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 16 16:53:56.175678 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:56.175630    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 16 16:53:56.176190 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:56.175699    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 16 16:53:57.175694 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:57.175648    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 16 16:53:57.176152 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:57.175716    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 16 16:53:58.169982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:58.169943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:58.179329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:58.179298    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:53:58.382970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:53:58.382938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:54:10.303552 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.303513    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:54:10.304078 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.303750    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" podUID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" containerName="maas-api" containerID="cri-o://67708ae70af047d4f5d6678a4521f5da9934f11bcaec35006cbb92c603edebff" gracePeriod=30
Sep 16 16:54:10.449519 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.449486    2577 generic.go:356] "Generic (PLEG): container finished" podID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" containerID="67708ae70af047d4f5d6678a4521f5da9934f11bcaec35006cbb92c603edebff" exitCode=0
Sep 16 16:54:10.449714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.449559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" event={"ID":"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d","Type":"ContainerDied","Data":"67708ae70af047d4f5d6678a4521f5da9934f11bcaec35006cbb92c603edebff"}
Sep 16 16:54:10.555961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.555875    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:54:10.649573 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.649532    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") pod \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") "
Sep 16 16:54:10.649755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.649583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca\") pod \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") "
Sep 16 16:54:10.649755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.649628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs\") pod \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") "
Sep 16 16:54:10.649755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.649671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcfw4\" (UniqueName: \"kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4\") pod \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\" (UID: \"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d\") "
Sep 16 16:54:10.649978 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.649955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" (UID: "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:10.650176 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.650150    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.651908 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.651882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" (UID: "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:54:10.652000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.651951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" (UID: "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:54:10.652000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.651951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4" (OuterVolumeSpecName: "kube-api-access-qcfw4") pod "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" (UID: "b73aac24-d043-48c0-9d84-2ecd4cfdfe6d"). InnerVolumeSpecName "kube-api-access-qcfw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:10.751681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.751638    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.751681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.751677    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.751681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:10.751690    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcfw4\" (UniqueName: \"kubernetes.io/projected/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d-kube-api-access-qcfw4\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:11.457107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.457067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"
Sep 16 16:54:11.457576 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.457114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2" event={"ID":"b73aac24-d043-48c0-9d84-2ecd4cfdfe6d","Type":"ContainerDied","Data":"8a0a8eed25765fca252b3bdab550bfc6b83a6d65197332445d89b0120b4a9164"}
Sep 16 16:54:11.457576 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.457177    2577 scope.go:117] "RemoveContainer" containerID="67708ae70af047d4f5d6678a4521f5da9934f11bcaec35006cbb92c603edebff"
Sep 16 16:54:11.478755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.478718    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:54:11.482674 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.482641    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1fd647e0-6b4646ddfb-fdsr2"]
Sep 16 16:54:11.653539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.653448    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:54:11.653756 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:11.653724    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" containerID="cri-o://07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b" gracePeriod=30
Sep 16 16:54:12.395214 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.395170    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:12.395746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.395727    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" containerName="maas-api"
Sep 16 16:54:12.395746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.395748    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" containerName="maas-api"
Sep 16 16:54:12.395921 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.395907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" containerName="maas-api"
Sep 16 16:54:12.399971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.399945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.403557 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.403530    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b2d51e27-openshift-default-dockercfg-7s9sz\""
Sep 16 16:54:12.411780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.411752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:12.579264 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579264 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpgg\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.579867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.579673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680324 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680324 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpgg\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680910 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.680910 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.681020 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.681020 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.680938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.681107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.681073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.681355 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.681338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.682910 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.682888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.683175 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.683148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.691999 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.691964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.692208 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.692156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpgg\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg\") pod \"e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") " pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:12.712174 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:12.712136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:13.007066 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.007035    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:13.008009 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:54:13.007982    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02633b3a_044e_4984_b466_f130e2cd26d3.slice/crio-9c480d18dbf7cc8f285895d1474032fd4d907ae38b5d09ac83670152fb8ee0bf WatchSource:0}: Error finding container 9c480d18dbf7cc8f285895d1474032fd4d907ae38b5d09ac83670152fb8ee0bf: Status 404 returned error can't find the container with id 9c480d18dbf7cc8f285895d1474032fd4d907ae38b5d09ac83670152fb8ee0bf
Sep 16 16:54:13.010374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.010332    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:13.010484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.010428    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:13.010484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.010459    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:13.384301 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.384260    2577 patch_prober.go:28] interesting pod/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 16 16:54:13.384508 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.384341    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:54:13.392082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.392049    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73aac24-d043-48c0-9d84-2ecd4cfdfe6d" path="/var/lib/kubelet/pods/b73aac24-d043-48c0-9d84-2ecd4cfdfe6d/volumes"
Sep 16 16:54:13.471979 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.471941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerStarted","Data":"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"}
Sep 16 16:54:13.472162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.472001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerStarted","Data":"9c480d18dbf7cc8f285895d1474032fd4d907ae38b5d09ac83670152fb8ee0bf"}
Sep 16 16:54:13.494275 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.494218    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podStartSLOduration=1.4942017619999999 podStartE2EDuration="1.494201762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:54:13.493831404 +0000 UTC m=+1070.579551480" watchObservedRunningTime="2026-09-16 16:54:13.494201762 +0000 UTC m=+1070.579921833"
Sep 16 16:54:13.713302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:13.713211    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:14.718177 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:14.718146    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:15.482983 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:15.482952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:15.483979 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:15.483960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:16.925767 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.925689    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935148    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935213    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935253    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935299    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935416    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935446    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.935493    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fk2r\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r\") pod \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\" (UID: \"309d8cac-a037-4b83-a3f8-9dbc9c25856a\") "
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.936185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.936358    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.936808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.936863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data" (OuterVolumeSpecName: "istio-data") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:16.937025 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.936998    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:16.940080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.940046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:16.940302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.940138    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:54:16.940302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.940216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token" (OuterVolumeSpecName: "istio-token") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:16.941826 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.941793    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r" (OuterVolumeSpecName: "kube-api-access-9fk2r") pod "309d8cac-a037-4b83-a3f8-9dbc9c25856a" (UID: "309d8cac-a037-4b83-a3f8-9dbc9c25856a"). InnerVolumeSpecName "kube-api-access-9fk2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:16.970490 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.970450    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:16.971033 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971014    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.971098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971036    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.971098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971046    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.971098 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971052    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.971197 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971132    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.971197 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.971139    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerName="istio-proxy"
Sep 16 16:54:16.975000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.974975    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:16.979196 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.979137    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:54:16.979196 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.979170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:54:16.979456 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.979315    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:54:16.979456 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.979350    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b2d51e27-serving-cert\""
Sep 16 16:54:16.979845 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.979827    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:54:16.995011 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:16.994974    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:17.036374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.036374 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrps6\" (UniqueName: \"kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036667    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036686    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036702    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fk2r\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-kube-api-access-9fk2r\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036717    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036730    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036742    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.036753 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036753    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.037073 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036765    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309d8cac-a037-4b83-a3f8-9dbc9c25856a-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.037073 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.036786    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309d8cac-a037-4b83-a3f8-9dbc9c25856a-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:17.137447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.137369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.137667 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.137497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.137667 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.137522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrps6\" (UniqueName: \"kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.137792 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.137664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.137792 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:17.137785    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-serving-cert: secret "maas-api-e2e-derive-b2d51e27-serving-cert" not found
Sep 16 16:54:17.137895 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:17.137861    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls podName:598d1d7f-585d-4453-ab60-163039e70bd4 nodeName:}" failed. No retries permitted until 2026-09-16 16:54:17.637840267 +0000 UTC m=+1074.723560332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls") pod "maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" (UID: "598d1d7f-585d-4453-ab60-163039e70bd4") : secret "maas-api-e2e-derive-b2d51e27-serving-cert" not found
Sep 16 16:54:17.138099 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.138075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.140068 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.140046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.148896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.148862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrps6\" (UniqueName: \"kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.497661 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.497626    2577 generic.go:356] "Generic (PLEG): container finished" podID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" containerID="07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b" exitCode=0
Sep 16 16:54:17.497869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.497685    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"
Sep 16 16:54:17.497869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.497716    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerDied","Data":"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"}
Sep 16 16:54:17.497869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.497786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4" event={"ID":"309d8cac-a037-4b83-a3f8-9dbc9c25856a","Type":"ContainerDied","Data":"2dfbd0fa09f86c29096acb54602d77f0db9804c2dbb8b8be536022aa4da19add"}
Sep 16 16:54:17.497869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.497819    2577 scope.go:117] "RemoveContainer" containerID="07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"
Sep 16 16:54:17.513815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.513794    2577 scope.go:117] "RemoveContainer" containerID="39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"
Sep 16 16:54:17.525118 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.525087    2577 scope.go:117] "RemoveContainer" containerID="07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"
Sep 16 16:54:17.525660 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:17.525623    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b\": container with ID starting with 07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b not found: ID does not exist" containerID="07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"
Sep 16 16:54:17.525845 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.525671    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b"} err="failed to get container status \"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b\": rpc error: code = NotFound desc = could not find container \"07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b\": container with ID starting with 07f50d447939cfb4086ba94b637c5bb29583e37c0f04286efcfd281aeede591b not found: ID does not exist"
Sep 16 16:54:17.525845 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.525697    2577 scope.go:117] "RemoveContainer" containerID="39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"
Sep 16 16:54:17.526086 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:17.526058    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2\": container with ID starting with 39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2 not found: ID does not exist" containerID="39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"
Sep 16 16:54:17.526161 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.526095    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2"} err="failed to get container status \"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2\": rpc error: code = NotFound desc = could not find container \"39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2\": container with ID starting with 39b1202634cdaf29e13c468d755ddcc034ae3401abc7ac4e3fb8f0564f0732d2 not found: ID does not exist"
Sep 16 16:54:17.545736 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.545700    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:54:17.563096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.563060    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1fd647e0-openshift-default-5d58657994-mkrj4"]
Sep 16 16:54:17.591105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.591069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:54:17.643514 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.643472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.646152 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.646126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") pod \"maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:17.886674 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:17.886619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:18.185988 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:18.185959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:18.188314 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:54:18.188270    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598d1d7f_585d_4453_ab60_163039e70bd4.slice/crio-af0955c0d98e8384dd94ae4123fc0376f1cad95191ddb6a943999a9aaed227d1 WatchSource:0}: Error finding container af0955c0d98e8384dd94ae4123fc0376f1cad95191ddb6a943999a9aaed227d1: Status 404 returned error can't find the container with id af0955c0d98e8384dd94ae4123fc0376f1cad95191ddb6a943999a9aaed227d1
Sep 16 16:54:18.506185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:18.506095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" event={"ID":"598d1d7f-585d-4453-ab60-163039e70bd4","Type":"ContainerStarted","Data":"af0955c0d98e8384dd94ae4123fc0376f1cad95191ddb6a943999a9aaed227d1"}
Sep 16 16:54:19.392327 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:19.392282    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309d8cac-a037-4b83-a3f8-9dbc9c25856a" path="/var/lib/kubelet/pods/309d8cac-a037-4b83-a3f8-9dbc9c25856a/volumes"
Sep 16 16:54:19.514224 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:19.514182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" event={"ID":"598d1d7f-585d-4453-ab60-163039e70bd4","Type":"ContainerStarted","Data":"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"}
Sep 16 16:54:19.514435 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:19.514314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:19.549634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:19.549571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" podStartSLOduration=3.093285624 podStartE2EDuration="3.54955239s" totalImagesPullingTime="456.266766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:54:19.54751798 +0000 UTC m=+1076.633238051" watchObservedRunningTime="2026-09-16 16:54:19.54955239 +0000 UTC m=+1076.635272466"
Sep 16 16:54:25.526807 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:25.526772    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:37.602120 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:37.602071    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:54:39.634612 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.634532    2577 generic.go:356] "Generic (PLEG): container finished" podID="02633b3a-044e-4984-b466-f130e2cd26d3" containerID="33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3" exitCode=0
Sep 16 16:54:39.635021 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.634604    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerDied","Data":"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"}
Sep 16 16:54:39.635100 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.635084    2577 scope.go:117] "RemoveContainer" containerID="33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"
Sep 16 16:54:39.636465 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.636430    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:39.636560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.636511    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:39.636560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:39.636545    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:54:40.646531 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:40.646483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerStarted","Data":"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"}
Sep 16 16:54:40.712357 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:40.712317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:40.713428 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:40.713373    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:40.713563 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:40.713456    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:41.713482 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:41.713440    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:41.713900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:41.713519    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:42.713375 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:42.713340    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:42.713375 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:42.713436    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:43.713310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:43.713272    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:43.713527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:43.713330    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:44.712668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:44.712621    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:44.713072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:44.712686    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:45.483942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:45.483892    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:45.713070 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:45.713025    2577 patch_prober.go:28] interesting pod/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 16 16:54:45.713617 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:45.713109    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 16 16:54:46.702018 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:46.701981    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:46.702284 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:46.702255    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" podUID="598d1d7f-585d-4453-ab60-163039e70bd4" containerName="maas-api" containerID="cri-o://30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f" gracePeriod=30
Sep 16 16:54:46.721489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:46.721452    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:46.723791 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:46.723765    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:46.959065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:46.958991    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:47.051250 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.051208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs\") pod \"598d1d7f-585d-4453-ab60-163039e70bd4\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") "
Sep 16 16:54:47.051558 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.051272    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrps6\" (UniqueName: \"kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6\") pod \"598d1d7f-585d-4453-ab60-163039e70bd4\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") "
Sep 16 16:54:47.051558 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.051484    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca\") pod \"598d1d7f-585d-4453-ab60-163039e70bd4\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") "
Sep 16 16:54:47.051558 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.051535    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") pod \"598d1d7f-585d-4453-ab60-163039e70bd4\" (UID: \"598d1d7f-585d-4453-ab60-163039e70bd4\") "
Sep 16 16:54:47.051885 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.051858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "598d1d7f-585d-4453-ab60-163039e70bd4" (UID: "598d1d7f-585d-4453-ab60-163039e70bd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:47.052089 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.052072    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598d1d7f-585d-4453-ab60-163039e70bd4-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.054468 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.054426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "598d1d7f-585d-4453-ab60-163039e70bd4" (UID: "598d1d7f-585d-4453-ab60-163039e70bd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:54:47.054671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.054491    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "598d1d7f-585d-4453-ab60-163039e70bd4" (UID: "598d1d7f-585d-4453-ab60-163039e70bd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:54:47.054671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.054575    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6" (OuterVolumeSpecName: "kube-api-access-xrps6") pod "598d1d7f-585d-4453-ab60-163039e70bd4" (UID: "598d1d7f-585d-4453-ab60-163039e70bd4"). InnerVolumeSpecName "kube-api-access-xrps6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:47.115506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.115465    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:47.153174 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.153127    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrps6\" (UniqueName: \"kubernetes.io/projected/598d1d7f-585d-4453-ab60-163039e70bd4-kube-api-access-xrps6\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.153174 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.153166    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.153174 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.153180    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598d1d7f-585d-4453-ab60-163039e70bd4-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.689526 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.689431    2577 generic.go:356] "Generic (PLEG): container finished" podID="598d1d7f-585d-4453-ab60-163039e70bd4" containerID="30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f" exitCode=0
Sep 16 16:54:47.689526 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.689483    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"
Sep 16 16:54:47.689743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.689520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" event={"ID":"598d1d7f-585d-4453-ab60-163039e70bd4","Type":"ContainerDied","Data":"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"}
Sep 16 16:54:47.689743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.689573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq" event={"ID":"598d1d7f-585d-4453-ab60-163039e70bd4","Type":"ContainerDied","Data":"af0955c0d98e8384dd94ae4123fc0376f1cad95191ddb6a943999a9aaed227d1"}
Sep 16 16:54:47.689743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.689598    2577 scope.go:117] "RemoveContainer" containerID="30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"
Sep 16 16:54:47.690101 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.690074    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" secret="" err="secret \"e2e-derive-b2d51e27-openshift-default-dockercfg-7s9sz\" not found"
Sep 16 16:54:47.699507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.699480    2577 scope.go:117] "RemoveContainer" containerID="30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"
Sep 16 16:54:47.699859 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:47.699836    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f\": container with ID starting with 30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f not found: ID does not exist" containerID="30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"
Sep 16 16:54:47.699912 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.699871    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f"} err="failed to get container status \"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f\": rpc error: code = NotFound desc = could not find container \"30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f\": container with ID starting with 30997a52a71c418f44bd99cd0a0be4e86daef5c9a44e5af740e7623b492e984f not found: ID does not exist"
Sep 16 16:54:47.719269 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.719230    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:47.737970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:47.737933    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2d51e27-6947fcd9d4-422xq"]
Sep 16 16:54:48.697118 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:48.697081    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy" containerID="cri-o://6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e" gracePeriod=30
Sep 16 16:54:49.392187 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:49.392143    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598d1d7f-585d-4453-ab60-163039e70bd4" path="/var/lib/kubelet/pods/598d1d7f-585d-4453-ab60-163039e70bd4/volumes"
Sep 16 16:54:53.306825 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:53.306777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:54:53.987142 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:53.987112    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:54.129336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129298    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlpgg\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129414    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129484    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129665    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:54.129883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129745    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket\") pod \"02633b3a-044e-4984-b466-f130e2cd26d3\" (UID: \"02633b3a-044e-4984-b466-f130e2cd26d3\") "
Sep 16 16:54:54.129883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.129867    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:54.130231 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.130205    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.130303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.130234    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.130659 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.130408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:54.130659 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.130446    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:54.131041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.131003    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:54.132964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.132757    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:54:54.132964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.132828    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg" (OuterVolumeSpecName: "kube-api-access-xlpgg") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "kube-api-access-xlpgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:54.132964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.132828    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:54.132964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.132917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02633b3a-044e-4984-b466-f130e2cd26d3" (UID: "02633b3a-044e-4984-b466-f130e2cd26d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231415    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlpgg\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-kube-api-access-xlpgg\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231453    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02633b3a-044e-4984-b466-f130e2cd26d3-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231464    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02633b3a-044e-4984-b466-f130e2cd26d3-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231472    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02633b3a-044e-4984-b466-f130e2cd26d3-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231481    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231489    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.231506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.231497    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02633b3a-044e-4984-b466-f130e2cd26d3-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:54:54.736329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.736291    2577 generic.go:356] "Generic (PLEG): container finished" podID="02633b3a-044e-4984-b466-f130e2cd26d3" containerID="6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e" exitCode=0
Sep 16 16:54:54.736857 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.736344    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"
Sep 16 16:54:54.736857 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.736375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerDied","Data":"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"}
Sep 16 16:54:54.736857 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.736463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs" event={"ID":"02633b3a-044e-4984-b466-f130e2cd26d3","Type":"ContainerDied","Data":"9c480d18dbf7cc8f285895d1474032fd4d907ae38b5d09ac83670152fb8ee0bf"}
Sep 16 16:54:54.736857 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.736498    2577 scope.go:117] "RemoveContainer" containerID="6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"
Sep 16 16:54:54.746675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.746654    2577 scope.go:117] "RemoveContainer" containerID="33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"
Sep 16 16:54:54.756189 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.756166    2577 scope.go:117] "RemoveContainer" containerID="6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"
Sep 16 16:54:54.756595 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:54.756571    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e\": container with ID starting with 6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e not found: ID does not exist" containerID="6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"
Sep 16 16:54:54.756685 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.756606    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e"} err="failed to get container status \"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e\": rpc error: code = NotFound desc = could not find container \"6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e\": container with ID starting with 6e6593f32529fcdaa70ac28512901916b6aa4f828c1ecfce7e7e6342097f255e not found: ID does not exist"
Sep 16 16:54:54.756685 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.756628    2577 scope.go:117] "RemoveContainer" containerID="33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"
Sep 16 16:54:54.756915 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:54:54.756900    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3\": container with ID starting with 33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3 not found: ID does not exist" containerID="33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"
Sep 16 16:54:54.756963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.756917    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3"} err="failed to get container status \"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3\": rpc error: code = NotFound desc = could not find container \"33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3\": container with ID starting with 33a615e945f9fabaa41bd3b986f226589786630066b3769dc3f0a9e5a13ef8a3 not found: ID does not exist"
Sep 16 16:54:54.758035 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.758011    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:54.762261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:54.762229    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b2d51e27-openshift-default-6f66f474c6-cndrs"]
Sep 16 16:54:55.391807 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:54:55.391759    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" path="/var/lib/kubelet/pods/02633b3a-044e-4984-b466-f130e2cd26d3/volumes"
Sep 16 16:55:05.527933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.527879    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:55:05.528398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528318    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:05.528398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528330    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:05.528398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528352    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598d1d7f-585d-4453-ab60-163039e70bd4" containerName="maas-api"
Sep 16 16:55:05.528398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528359    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="598d1d7f-585d-4453-ab60-163039e70bd4" containerName="maas-api"
Sep 16 16:55:05.528539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528462    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="598d1d7f-585d-4453-ab60-163039e70bd4" containerName="maas-api"
Sep 16 16:55:05.528539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528473    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:05.528539 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.528480    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:05.533213 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.533186    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.538017 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.537990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e2ea3e\"/\"test-model-e2ea3e-kserve-self-signed-certs\""
Sep 16 16:55:05.538185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.538023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e2ea3e\"/\"openshift-service-ca.crt\""
Sep 16 16:55:05.538185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.538070    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e2ea3e\"/\"kube-root-ca.crt\""
Sep 16 16:55:05.539074 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.539058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e2ea3e\"/\"default-dockercfg-4625k\""
Sep 16 16:55:05.566081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.566044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:55:05.651081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.651280 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.651280 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs5dp\" (UniqueName: \"kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.651280 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.651461 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.651461 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.651320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.752876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.752827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.752876 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.752891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.753162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.752954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.753162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.753037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.753162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.753068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs5dp\" (UniqueName: \"kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.753162 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.753144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.753734 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.753709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.754169 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.754112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.754450 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.754423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.760230 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.756752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.760230 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.757045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.768331 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.768298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs5dp\" (UniqueName: \"kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp\") pod \"test-model-e2ea3e-kserve-79f794b8cf-znx6w\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") " pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:05.842951 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:05.842891    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:06.127597 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:06.127508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:55:06.130221 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:55:06.130192    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e31d280_7ca0_44dd_a122_6914159ea17b.slice/crio-87e661d3001c050fa6ed44cb623e15239f15eac638898e9c83e52bc10cb7778f WatchSource:0}: Error finding container 87e661d3001c050fa6ed44cb623e15239f15eac638898e9c83e52bc10cb7778f: Status 404 returned error can't find the container with id 87e661d3001c050fa6ed44cb623e15239f15eac638898e9c83e52bc10cb7778f
Sep 16 16:55:06.811015 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:06.810972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" event={"ID":"4e31d280-7ca0-44dd-a122-6914159ea17b","Type":"ContainerStarted","Data":"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"}
Sep 16 16:55:06.811470 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:06.811027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" event={"ID":"4e31d280-7ca0-44dd-a122-6914159ea17b","Type":"ContainerStarted","Data":"87e661d3001c050fa6ed44cb623e15239f15eac638898e9c83e52bc10cb7778f"}
Sep 16 16:55:06.832546 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:06.832482    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" podStartSLOduration=1.83246117 podStartE2EDuration="1.83246117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:55:06.830426399 +0000 UTC m=+1123.916146468" watchObservedRunningTime="2026-09-16 16:55:06.83246117 +0000 UTC m=+1123.918181244"
Sep 16 16:55:11.808764 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:11.808725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:55:13.285515 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.285476    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"]
Sep 16 16:55:13.285919 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.285909    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:13.285971 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.285920    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="02633b3a-044e-4984-b466-f130e2cd26d3" containerName="istio-proxy"
Sep 16 16:55:13.289729 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.289704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:13.292132 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.292110    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:55:13.292254 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.292147    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:55:13.299406 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.298830    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"]
Sep 16 16:55:13.437540 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.437497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-708c6a-j2glw\" (UID: \"c3191331-8758-469a-8e1f-842e7252cd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:13.539226 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.539180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-708c6a-j2glw\" (UID: \"c3191331-8758-469a-8e1f-842e7252cd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:13.539854 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.539831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-708c6a-j2glw\" (UID: \"c3191331-8758-469a-8e1f-842e7252cd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:13.610803 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.610759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:13.886044 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:13.885984    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"]
Sep 16 16:55:13.888682 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:55:13.888649    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3191331_8758_469a_8e1f_842e7252cd0b.slice/crio-003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d WatchSource:0}: Error finding container 003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d: Status 404 returned error can't find the container with id 003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d
Sep 16 16:55:14.857891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:14.857846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw" event={"ID":"c3191331-8758-469a-8e1f-842e7252cd0b","Type":"ContainerStarted","Data":"003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d"}
Sep 16 16:55:15.843545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:15.843504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:15.843743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:15.843568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:15.863845 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:15.863813    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:15.878840 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:15.878798    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:55:16.878730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:16.878688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw" event={"ID":"c3191331-8758-469a-8e1f-842e7252cd0b","Type":"ContainerStarted","Data":"6f268699d23084ff4ccdbac1c459ed2685251bbc3a7ac6d0cdb50550ca3bc0c5"}
Sep 16 16:55:16.898120 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:16.898066    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw" podStartSLOduration=1.5514711650000002 podStartE2EDuration="3.898050836s" totalImagesPullingTime="2.346579671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:55:16.896311477 +0000 UTC m=+1133.982031549" watchObservedRunningTime="2026-09-16 16:55:16.898050836 +0000 UTC m=+1133.983770910"
Sep 16 16:55:17.886055 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:17.886015    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3191331-8758-469a-8e1f-842e7252cd0b" containerID="6f268699d23084ff4ccdbac1c459ed2685251bbc3a7ac6d0cdb50550ca3bc0c5" exitCode=0
Sep 16 16:55:17.886521 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:17.886076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw" event={"ID":"c3191331-8758-469a-8e1f-842e7252cd0b","Type":"ContainerDied","Data":"6f268699d23084ff4ccdbac1c459ed2685251bbc3a7ac6d0cdb50550ca3bc0c5"}
Sep 16 16:55:19.046551 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.046520    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:19.203566 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.203456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca\") pod \"c3191331-8758-469a-8e1f-842e7252cd0b\" (UID: \"c3191331-8758-469a-8e1f-842e7252cd0b\") "
Sep 16 16:55:19.203850 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.203823    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3191331-8758-469a-8e1f-842e7252cd0b" (UID: "c3191331-8758-469a-8e1f-842e7252cd0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:19.203965 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.203945    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3191331-8758-469a-8e1f-842e7252cd0b-maas-api-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:19.900991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.900939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw" event={"ID":"c3191331-8758-469a-8e1f-842e7252cd0b","Type":"ContainerDied","Data":"003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d"}
Sep 16 16:55:19.901173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.901011    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="003142038ffddfe1b2f82b010438131fe5c918565bed27804bdcf70168d9b10d"
Sep 16 16:55:19.901173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:19.900947    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"
Sep 16 16:55:24.242578 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:24.242533    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"]
Sep 16 16:55:24.246061 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:24.246023    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-708c6a-j2glw"]
Sep 16 16:55:25.394640 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:25.394602    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3191331-8758-469a-8e1f-842e7252cd0b" path="/var/lib/kubelet/pods/c3191331-8758-469a-8e1f-842e7252cd0b/volumes"
Sep 16 16:55:28.678833 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:28.678797    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:55:31.774536 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:31.774501    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:55:39.866007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:39.865963    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:55:44.276266 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:44.276229    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:55:44.276828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:44.276590    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" containerID="cri-o://d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c" gracePeriod=30
Sep 16 16:55:45.874156 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:45.874070    2577 patch_prober.go:28] interesting pod/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:55:45.874572 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:45.874176    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:55:46.795243 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:46.795194    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" podUID="745834cc-3e2a-4984-8ceb-50a48068c0ec" containerName="main" containerID="cri-o://b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81" gracePeriod=45
Sep 16 16:55:47.050280 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.050210    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:55:47.072711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.072678    2577 generic.go:356] "Generic (PLEG): container finished" podID="745834cc-3e2a-4984-8ceb-50a48068c0ec" containerID="b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81" exitCode=0
Sep 16 16:55:47.072933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.072733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"
Sep 16 16:55:47.072933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.072762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" event={"ID":"745834cc-3e2a-4984-8ceb-50a48068c0ec","Type":"ContainerDied","Data":"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"}
Sep 16 16:55:47.072933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.072821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc" event={"ID":"745834cc-3e2a-4984-8ceb-50a48068c0ec","Type":"ContainerDied","Data":"cc924dba82f433f4bb28abb3d2b22f933f7c0453ac96d72f0d10d3ee45755cee"}
Sep 16 16:55:47.072933 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.072844    2577 scope.go:117] "RemoveContainer" containerID="b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"
Sep 16 16:55:47.084214 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.084187    2577 scope.go:117] "RemoveContainer" containerID="b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"
Sep 16 16:55:47.084682 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:55:47.084657    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81\": container with ID starting with b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81 not found: ID does not exist" containerID="b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"
Sep 16 16:55:47.084828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.084693    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81"} err="failed to get container status \"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81\": rpc error: code = NotFound desc = could not find container \"b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81\": container with ID starting with b346ced76d2584b2d9b1b3301cc3faaf2d54a24379997fcea825d7f04407ff81 not found: ID does not exist"
Sep 16 16:55:47.205522 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205484    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205522 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jp6v\" (UniqueName: \"kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205597    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205774 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs\") pod \"745834cc-3e2a-4984-8ceb-50a48068c0ec\" (UID: \"745834cc-3e2a-4984-8ceb-50a48068c0ec\") "
Sep 16 16:55:47.205990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:47.205990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205876    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home" (OuterVolumeSpecName: "home") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:47.205990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.205895    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:47.206148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.206101    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.206148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.206121    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.206148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.206132    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.208037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.208010    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v" (OuterVolumeSpecName: "kube-api-access-8jp6v") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "kube-api-access-8jp6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:47.208292 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.208271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:55:47.208358 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.208273    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm" (OuterVolumeSpecName: "dshm") pod "745834cc-3e2a-4984-8ceb-50a48068c0ec" (UID: "745834cc-3e2a-4984-8ceb-50a48068c0ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:47.307795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.307685    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745834cc-3e2a-4984-8ceb-50a48068c0ec-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.307795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.307725    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jp6v\" (UniqueName: \"kubernetes.io/projected/745834cc-3e2a-4984-8ceb-50a48068c0ec-kube-api-access-8jp6v\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.307795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.307736    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745834cc-3e2a-4984-8ceb-50a48068c0ec-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:47.411195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.411149    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:55:47.424110 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:47.424072    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-eb2ab2/simulator-w0-eb2ab2-kserve-6dfb5d7b67-gmmmc"]
Sep 16 16:55:49.391722 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.391690    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="745834cc-3e2a-4984-8ceb-50a48068c0ec" path="/var/lib/kubelet/pods/745834cc-3e2a-4984-8ceb-50a48068c0ec/volumes"
Sep 16 16:55:49.547647 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.547571    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:55:49.636564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7gjw\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636890 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636626    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636890 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636655    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636890 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636890 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.636890 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636858    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.637149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636898    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.637149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.636939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket\") pod \"03015ef9-3088-43b8-b16f-35d12421533a\" (UID: \"03015ef9-3088-43b8-b16f-35d12421533a\") "
Sep 16 16:55:49.637251 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637148    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:49.637313 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637256    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:49.637415 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:49.637480 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637410    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:49.637529 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637484    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data" (OuterVolumeSpecName: "istio-data") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:49.637563 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637541    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.637601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637564    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.637601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637579    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.637601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.637594    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03015ef9-3088-43b8-b16f-35d12421533a-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.639009 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.638985    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:55:49.639133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.639015    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:49.639297 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.639278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw" (OuterVolumeSpecName: "kube-api-access-s7gjw") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "kube-api-access-s7gjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:49.639363 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.639304    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token" (OuterVolumeSpecName: "istio-token") pod "03015ef9-3088-43b8-b16f-35d12421533a" (UID: "03015ef9-3088-43b8-b16f-35d12421533a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:49.738304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.738257    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7gjw\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-kube-api-access-s7gjw\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.738304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.738296    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03015ef9-3088-43b8-b16f-35d12421533a-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.738304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.738308    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.738304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.738316    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03015ef9-3088-43b8-b16f-35d12421533a-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:49.738622 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:49.738326    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03015ef9-3088-43b8-b16f-35d12421533a-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:50.092572 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.092538    2577 generic.go:356] "Generic (PLEG): container finished" podID="03015ef9-3088-43b8-b16f-35d12421533a" containerID="d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c" exitCode=0
Sep 16 16:55:50.092747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.092590    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"
Sep 16 16:55:50.092747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.092622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerDied","Data":"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"}
Sep 16 16:55:50.092747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.092678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq" event={"ID":"03015ef9-3088-43b8-b16f-35d12421533a","Type":"ContainerDied","Data":"39cf981b35d2ca795607a1379712152c6f0e0e7ce3d9fd3b5985c82151af2a5c"}
Sep 16 16:55:50.092747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.092700    2577 scope.go:117] "RemoveContainer" containerID="d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"
Sep 16 16:55:50.102682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.102660    2577 scope.go:117] "RemoveContainer" containerID="657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"
Sep 16 16:55:50.111635 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.111610    2577 scope.go:117] "RemoveContainer" containerID="d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"
Sep 16 16:55:50.112009 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:55:50.111987    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c\": container with ID starting with d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c not found: ID does not exist" containerID="d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"
Sep 16 16:55:50.112082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.112024    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c"} err="failed to get container status \"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c\": rpc error: code = NotFound desc = could not find container \"d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c\": container with ID starting with d7e6595362c8279fabe4f0c949c424439fe6daa8685b4047b446c622819efa4c not found: ID does not exist"
Sep 16 16:55:50.112082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.112054    2577 scope.go:117] "RemoveContainer" containerID="657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"
Sep 16 16:55:50.112347 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:55:50.112329    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87\": container with ID starting with 657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87 not found: ID does not exist" containerID="657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"
Sep 16 16:55:50.112405 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.112354    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87"} err="failed to get container status \"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87\": rpc error: code = NotFound desc = could not find container \"657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87\": container with ID starting with 657fdb38e8b0fa04c945138bb026049ce65079df577c9f7966d040053a809d87 not found: ID does not exist"
Sep 16 16:55:50.118043 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.118011    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:55:50.124681 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:50.124641    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-eb2ab2-openshift-default-568c464d88-6vbnq"]
Sep 16 16:55:51.214208 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:51.214158    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:55:51.392277 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:51.392242    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03015ef9-3088-43b8-b16f-35d12421533a" path="/var/lib/kubelet/pods/03015ef9-3088-43b8-b16f-35d12421533a/volumes"
Sep 16 16:55:54.886242 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:54.886199    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" podUID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" containerName="main" containerID="cri-o://e5f530be8de060a616b772700346a70542b37f6ed7eacdf54be3e38f87671f1c" gracePeriod=45
Sep 16 16:55:55.129901 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.129867    2577 generic.go:356] "Generic (PLEG): container finished" podID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" containerID="e5f530be8de060a616b772700346a70542b37f6ed7eacdf54be3e38f87671f1c" exitCode=0
Sep 16 16:55:55.130062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.129943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" event={"ID":"047ca784-4c2e-4ab3-80f9-44cbca0c75c2","Type":"ContainerDied","Data":"e5f530be8de060a616b772700346a70542b37f6ed7eacdf54be3e38f87671f1c"}
Sep 16 16:55:55.130062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.129999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9" event={"ID":"047ca784-4c2e-4ab3-80f9-44cbca0c75c2","Type":"ContainerDied","Data":"41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41"}
Sep 16 16:55:55.130062 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.130012    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41d4b2e4b365dfd0f544f747d77cb2726206e665c220334bf8b18102907bfa41"
Sep 16 16:55:55.133645 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.133614    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:55:55.298889 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.298854    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.298920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqd72\" (UniqueName: \"kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.298952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.298982    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299080 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299377 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299083    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs\") pod \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\" (UID: \"047ca784-4c2e-4ab3-80f9-44cbca0c75c2\") "
Sep 16 16:55:55.299377 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:55.299377 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home" (OuterVolumeSpecName: "home") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:55.299377 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299306    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:55.299630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299514    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:55.299630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299528    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:55.299630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.299536    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:55.301476 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.301439    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm" (OuterVolumeSpecName: "dshm") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:55:55.301476 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.301475    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72" (OuterVolumeSpecName: "kube-api-access-jqd72") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "kube-api-access-jqd72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:55.301664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.301485    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "047ca784-4c2e-4ab3-80f9-44cbca0c75c2" (UID: "047ca784-4c2e-4ab3-80f9-44cbca0c75c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:55:55.400183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.400154    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqd72\" (UniqueName: \"kubernetes.io/projected/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-kube-api-access-jqd72\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:55.400306 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.400184    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:55.400306 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:55.400194    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/047ca784-4c2e-4ab3-80f9-44cbca0c75c2-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:55:56.135488 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:56.135450    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"
Sep 16 16:55:56.150752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:56.150708    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:55:56.154883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:56.154848    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f604b4/simulator-w4-f604b4-kserve-b78c6d699-r2qz9"]
Sep 16 16:55:57.391022 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:55:57.390980    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" path="/var/lib/kubelet/pods/047ca784-4c2e-4ab3-80f9-44cbca0c75c2/volumes"
Sep 16 16:56:03.612791 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:03.612747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:56:07.041300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.041254    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:56:07.041827 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.041609    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" containerID="cri-o://6e23325a897fb40cea94f2bba9367c9a8462e13f95e52e9d6413b9fdf5bf6c04" gracePeriod=30
Sep 16 16:56:07.770887 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.770852    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:56:07.771311 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771287    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:07.771311 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771310    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771323    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771332    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771349    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3191331-8758-469a-8e1f-842e7252cd0b" containerName="revoke-keys"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771358    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3191331-8758-469a-8e1f-842e7252cd0b" containerName="revoke-keys"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771411    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745834cc-3e2a-4984-8ceb-50a48068c0ec" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771418    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="745834cc-3e2a-4984-8ceb-50a48068c0ec" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771518    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771529    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771536    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="745834cc-3e2a-4984-8ceb-50a48068c0ec" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771545    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="047ca784-4c2e-4ab3-80f9-44cbca0c75c2" containerName="main"
Sep 16 16:56:07.771580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.771553    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3191331-8758-469a-8e1f-842e7252cd0b" containerName="revoke-keys"
Sep 16 16:56:07.776259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.776233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.779095 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.779065    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-97b645-openshift-default-dockercfg-p6vqh\""
Sep 16 16:56:07.794273 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.794239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:56:07.826206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826712 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826712 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.826712 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.826667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7p9\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7p9\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927469 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927885 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927885 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.927885 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.928041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.928041 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.927980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.928250 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.928227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.929797 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.929774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.930015 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.929996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.935961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.935930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:07.936079 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:07.936050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7p9\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9\") pod \"e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") " pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:08.087901 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.087856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:08.368349 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.368256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:56:08.371849 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:56:08.371815    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29454103_f4b9_4ba1_9264_b514ec1483aa.slice/crio-1646f63844561caeceabbab06425d5e26a971ee8379c1ef8f84048071e4c4e42 WatchSource:0}: Error finding container 1646f63844561caeceabbab06425d5e26a971ee8379c1ef8f84048071e4c4e42: Status 404 returned error can't find the container with id 1646f63844561caeceabbab06425d5e26a971ee8379c1ef8f84048071e4c4e42
Sep 16 16:56:08.374236 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.374193    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:08.374345 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.374267    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:08.374345 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.374299    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:08.933000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.932953    2577 patch_prober.go:28] interesting pod/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:56:08.933185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:08.933025    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:56:09.210234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:09.210137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerStarted","Data":"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"}
Sep 16 16:56:09.210234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:09.210203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerStarted","Data":"1646f63844561caeceabbab06425d5e26a971ee8379c1ef8f84048071e4c4e42"}
Sep 16 16:56:09.233536 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:09.233471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podStartSLOduration=2.233451269 podStartE2EDuration="2.233451269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:09.230782958 +0000 UTC m=+1186.316503032" watchObservedRunningTime="2026-09-16 16:56:09.233451269 +0000 UTC m=+1186.319171342"
Sep 16 16:56:10.088194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:10.088151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:10.093493 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:10.093457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:10.215445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:10.215416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:10.216397 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:10.216369    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:12.240752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.240714    2577 generic.go:356] "Generic (PLEG): container finished" podID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerID="6e23325a897fb40cea94f2bba9367c9a8462e13f95e52e9d6413b9fdf5bf6c04" exitCode=0
Sep 16 16:56:12.241329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.240750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerDied","Data":"6e23325a897fb40cea94f2bba9367c9a8462e13f95e52e9d6413b9fdf5bf6c04"}
Sep 16 16:56:12.241587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.241427    2577 scope.go:117] "RemoveContainer" containerID="0fddc0dfa1ca5670771450002b01d1f4ec47b08e7e79ceeeea20becfdf8607dd"
Sep 16 16:56:12.306246 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.306168    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:56:12.377916 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.377872    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6prwn\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.377939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.377969    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378059    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy\") pod \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\" (UID: \"ec6990ec-3c4f-4a43-9690-6d443d870e0a\") "
Sep 16 16:56:12.378560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:56:12.378560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378456    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:56:12.378560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378544    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:56:12.378727 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378677    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:56:12.378727 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378689    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:56:12.378843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378766    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.378843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378782    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.378843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378790    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.378843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378800    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.378843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.378808    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.380479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.380449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:56:12.380619 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.380476    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:56:12.380714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.380661    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:56:12.380714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.380684    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn" (OuterVolumeSpecName: "kube-api-access-6prwn") pod "ec6990ec-3c4f-4a43-9690-6d443d870e0a" (UID: "ec6990ec-3c4f-4a43-9690-6d443d870e0a"). InnerVolumeSpecName "kube-api-access-6prwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:56:12.479794 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.479753    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6prwn\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-kube-api-access-6prwn\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.479794 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.479792    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.480027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.479806    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:12.480027 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:12.479817    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec6990ec-3c4f-4a43-9690-6d443d870e0a-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:56:13.248173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.248137    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"
Sep 16 16:56:13.248173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.248168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8" event={"ID":"ec6990ec-3c4f-4a43-9690-6d443d870e0a","Type":"ContainerDied","Data":"0f05cd2cbef6480019643c68a1ed5b439b2cf8aeb5b79b282cd33a2a9e938bb9"}
Sep 16 16:56:13.248737 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.248228    2577 scope.go:117] "RemoveContainer" containerID="6e23325a897fb40cea94f2bba9367c9a8462e13f95e52e9d6413b9fdf5bf6c04"
Sep 16 16:56:13.277099 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.277059    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:56:13.286648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.286612    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-708c6a-openshift-default-84d74f76c6-n8gf8"]
Sep 16 16:56:13.392525 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.392485    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" path="/var/lib/kubelet/pods/ec6990ec-3c4f-4a43-9690-6d443d870e0a/volumes"
Sep 16 16:56:13.571013 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.570970    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:56:13.571591 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571569    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:13.571591 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571591    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:13.571762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571629    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:13.571762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571637    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="03015ef9-3088-43b8-b16f-35d12421533a" containerName="istio-proxy"
Sep 16 16:56:13.571762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571708    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:13.571762 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.571719    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:13.576698 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.576673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.579434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.579411    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:56:13.579434 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.579428    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:56:13.579700 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.579682    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-97b645-serving-cert\""
Sep 16 16:56:13.579887 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.579871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:56:13.580119 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.580103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:56:13.587697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.586521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:56:13.693848 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.693811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjbjb\" (UniqueName: \"kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.694050 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.693891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.694050 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.693924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.694050 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.693964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.795227 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.795187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.795227 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.795237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.795562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.795268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.795562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.795338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjbjb\" (UniqueName: \"kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.795976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.795952    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.797906 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.797882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.798060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.798043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.803716 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.803691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjbjb\" (UniqueName: \"kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb\") pod \"maas-api-e2e-explicit-97b645-5554b95b96-fk2js\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:13.893761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:13.893660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:14.185982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:14.185947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:56:14.188554 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:56:14.188513    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906d2c00_dfb6_4d0d_b6f4_ff701389d751.slice/crio-bf6ac6a9a2909a5487dfb5ac006e7ccf7c45a3b5691bc11d7a27f05d7634dbce WatchSource:0}: Error finding container bf6ac6a9a2909a5487dfb5ac006e7ccf7c45a3b5691bc11d7a27f05d7634dbce: Status 404 returned error can't find the container with id bf6ac6a9a2909a5487dfb5ac006e7ccf7c45a3b5691bc11d7a27f05d7634dbce
Sep 16 16:56:14.256510 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:14.256475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" event={"ID":"906d2c00-dfb6-4d0d-b6f4-ff701389d751","Type":"ContainerStarted","Data":"bf6ac6a9a2909a5487dfb5ac006e7ccf7c45a3b5691bc11d7a27f05d7634dbce"}
Sep 16 16:56:15.264048 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:15.264013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" event={"ID":"906d2c00-dfb6-4d0d-b6f4-ff701389d751","Type":"ContainerStarted","Data":"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"}
Sep 16 16:56:15.264513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:15.264107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:15.283237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:15.283178    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" podStartSLOduration=1.8910587479999998 podStartE2EDuration="2.283159212s" totalImagesPullingTime="392.100464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:15.280607325 +0000 UTC m=+1192.366327399" watchObservedRunningTime="2026-09-16 16:56:15.283159212 +0000 UTC m=+1192.368879284"
Sep 16 16:56:21.276851 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:21.276815    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:56:21.319720 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:21.319680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:56:25.268320 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.268286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:56:25.268797 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.268768    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:25.268797 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.268783    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec6990ec-3c4f-4a43-9690-6d443d870e0a" containerName="istio-proxy"
Sep 16 16:56:25.274413 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.274262    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.281404 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.279759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f3a3973e-openshift-default-dockercfg-hcvks\""
Sep 16 16:56:25.288422 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.288371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:56:25.426236 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426481 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8g97\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426481 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426625 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426625 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426747 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.426923 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.426879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.528111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.528007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.528443 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.528415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.528664 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.528639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8g97\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.529475 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.529449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.529670 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.529652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.529790 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.529759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.529991 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.529801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530139 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.530122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.530245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530364 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.530065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530493 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.530468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530558 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.529097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.530686 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.530668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.531221 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.531192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.531647 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.531624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.532990 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.532943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.538793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.538762    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.541146 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.541116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8g97\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97\") pod \"e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") " pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.596260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.596216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:25.937252 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.937205    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:56:25.948601 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.948554    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:25.948743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.948672    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:25.948743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:25.948737    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:26.353834 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:26.353791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerStarted","Data":"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"}
Sep 16 16:56:26.354300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:26.353852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerStarted","Data":"ecf6108967fc8b866cfe15960d3a273d9e935d91b3e9a880594814cee0817c05"}
Sep 16 16:56:26.379274 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:26.379194    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podStartSLOduration=1.379170991 podStartE2EDuration="1.379170991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:26.378652011 +0000 UTC m=+1203.464372100" watchObservedRunningTime="2026-09-16 16:56:26.379170991 +0000 UTC m=+1203.464891064"
Sep 16 16:56:26.596504 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:26.596470    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:27.601877 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:27.601846    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:28.364951 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:28.364911    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:28.366109 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:28.366088    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:32.388606 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.388571    2577 generic.go:356] "Generic (PLEG): container finished" podID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerID="5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f" exitCode=0
Sep 16 16:56:32.389068 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.388642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerDied","Data":"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"}
Sep 16 16:56:32.389116 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.389085    2577 scope.go:117] "RemoveContainer" containerID="5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"
Sep 16 16:56:32.390275 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.390243    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:32.390347 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.390315    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:32.390401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:32.390350    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:33.396224 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:33.396174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerStarted","Data":"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"}
Sep 16 16:56:34.088919 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:34.088875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:34.089225 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:34.089200    2577 patch_prober.go:28] interesting pod/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 16 16:56:34.089275 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:34.089254    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 16 16:56:35.094719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:35.094671    2577 patch_prober.go:28] interesting pod/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 16 16:56:35.095237 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:35.094741    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 16 16:56:36.088514 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:36.088470    2577 patch_prober.go:28] interesting pod/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 16 16:56:36.088713 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:36.088530    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 16 16:56:37.088866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:37.088823    2577 patch_prober.go:28] interesting pod/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 16 16:56:37.089399 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:37.088894    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 16 16:56:38.088976 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:38.088933    2577 patch_prober.go:28] interesting pod/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 16 16:56:38.089425 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:38.088991    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 16 16:56:39.093037 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:39.092999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:39.093537 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:39.093316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:39.094047 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:39.094031    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:56:41.609000 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:41.608956    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:56:52.515773 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.515735    2577 generic.go:356] "Generic (PLEG): container finished" podID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerID="b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6" exitCode=0
Sep 16 16:56:52.516353 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.515811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerDied","Data":"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"}
Sep 16 16:56:52.516445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.516427    2577 scope.go:117] "RemoveContainer" containerID="b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"
Sep 16 16:56:52.517663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.517631    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:52.517741 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.517706    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:52.517784 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:52.517740    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:56:53.526040 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:53.525002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerStarted","Data":"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"}
Sep 16 16:56:53.596671 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:53.596627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:53.597815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:53.597782    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:53.597964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:53.597839    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:54.596789 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:54.596755    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:54.597225 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:54.596833    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:55.596633 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:55.596596    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:55.596795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:55.596658    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:56.553454 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.553421    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:56:56.556471 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.556444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.570052 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.570030    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-97b645\"/\"kube-root-ca.crt\""
Sep 16 16:56:56.570179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.570032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-97b645\"/\"explicit-model-97b645-kserve-self-signed-certs\""
Sep 16 16:56:56.570179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.570153    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-97b645\"/\"default-dockercfg-npdjr\""
Sep 16 16:56:56.570254 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.570197    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-97b645\"/\"openshift-service-ca.crt\""
Sep 16 16:56:56.597018 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.596983    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:56.597195 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.597054    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:56.609086 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.609051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:56:56.712897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.712862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l5l5\" (UniqueName: \"kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.712897 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.712900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.713151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.712937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.713151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.713039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.713151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.713070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.713265 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.713152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.813862 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.813771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l5l5\" (UniqueName: \"kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814092 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814213 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814259 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814344 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814572 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814629 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.814679 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.814609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.816760 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.816732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.817148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.817127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.846082 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.846037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l5l5\" (UniqueName: \"kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5\") pod \"explicit-model-97b645-kserve-58db844675-sdzq8\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") " pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:56.866035 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:56.865992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:56:57.168773 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.168738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:56:57.170192 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:56:57.170166    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode747ab49_4b34_4cbe_bd89_dfba2b02a381.slice/crio-cf6d2cac11b87688c4495f1c6de72bb89fe496e62257ace85deb87b905656bed WatchSource:0}: Error finding container cf6d2cac11b87688c4495f1c6de72bb89fe496e62257ace85deb87b905656bed: Status 404 returned error can't find the container with id cf6d2cac11b87688c4495f1c6de72bb89fe496e62257ace85deb87b905656bed
Sep 16 16:56:57.561815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.561779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" event={"ID":"e747ab49-4b34-4cbe-bd89-dfba2b02a381","Type":"ContainerStarted","Data":"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"}
Sep 16 16:56:57.562234 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.561822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" event={"ID":"e747ab49-4b34-4cbe-bd89-dfba2b02a381","Type":"ContainerStarted","Data":"cf6d2cac11b87688c4495f1c6de72bb89fe496e62257ace85deb87b905656bed"}
Sep 16 16:56:57.588489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.588435    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" podStartSLOduration=1.588416714 podStartE2EDuration="1.588416714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:57.587415219 +0000 UTC m=+1234.673135291" watchObservedRunningTime="2026-09-16 16:56:57.588416714 +0000 UTC m=+1234.674136787"
Sep 16 16:56:57.596775 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.596745    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:57.596938 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:57.596807    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:58.364952 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:58.364910    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:58.597720 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:58.597679    2577 patch_prober.go:28] interesting pod/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 16 16:56:58.598153 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:58.597755    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 16 16:56:59.279577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:59.279531    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:56:59.602457 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:59.602424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:56:59.603571 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:56:59.603550    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:57:06.866970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:06.866929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:57:08.997274 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:08.997236    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:57:10.307206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.307164    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:57:10.307736 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.307458    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" podUID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" containerName="maas-api" containerID="cri-o://73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f" gracePeriod=30
Sep 16 16:57:10.573565 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.573467    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:57:10.637580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.637540    2577 generic.go:356] "Generic (PLEG): container finished" podID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" containerID="73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f" exitCode=0
Sep 16 16:57:10.637580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.637581    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"
Sep 16 16:57:10.637809 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.637627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" event={"ID":"906d2c00-dfb6-4d0d-b6f4-ff701389d751","Type":"ContainerDied","Data":"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"}
Sep 16 16:57:10.637809 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.637680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js" event={"ID":"906d2c00-dfb6-4d0d-b6f4-ff701389d751","Type":"ContainerDied","Data":"bf6ac6a9a2909a5487dfb5ac006e7ccf7c45a3b5691bc11d7a27f05d7634dbce"}
Sep 16 16:57:10.637809 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.637707    2577 scope.go:117] "RemoveContainer" containerID="73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"
Sep 16 16:57:10.647703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.647675    2577 scope.go:117] "RemoveContainer" containerID="73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"
Sep 16 16:57:10.648073 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:10.648048    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f\": container with ID starting with 73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f not found: ID does not exist" containerID="73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"
Sep 16 16:57:10.648145 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.648086    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f"} err="failed to get container status \"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f\": rpc error: code = NotFound desc = could not find container \"73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f\": container with ID starting with 73211a39415feeeb589271d6a2e92a1f3dce3d8b9f53c198b2179cdaa861275f not found: ID does not exist"
Sep 16 16:57:10.772303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.772265    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs\") pod \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") "
Sep 16 16:57:10.772549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.772312    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjbjb\" (UniqueName: \"kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb\") pod \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") "
Sep 16 16:57:10.772549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.772401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls\") pod \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") "
Sep 16 16:57:10.772549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.772434    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca\") pod \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\" (UID: \"906d2c00-dfb6-4d0d-b6f4-ff701389d751\") "
Sep 16 16:57:10.772873 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.772834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "906d2c00-dfb6-4d0d-b6f4-ff701389d751" (UID: "906d2c00-dfb6-4d0d-b6f4-ff701389d751"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:10.774785 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.774758    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "906d2c00-dfb6-4d0d-b6f4-ff701389d751" (UID: "906d2c00-dfb6-4d0d-b6f4-ff701389d751"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:57:10.774905 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.774786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "906d2c00-dfb6-4d0d-b6f4-ff701389d751" (UID: "906d2c00-dfb6-4d0d-b6f4-ff701389d751"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:57:10.774905 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.774802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb" (OuterVolumeSpecName: "kube-api-access-bjbjb") pod "906d2c00-dfb6-4d0d-b6f4-ff701389d751" (UID: "906d2c00-dfb6-4d0d-b6f4-ff701389d751"). InnerVolumeSpecName "kube-api-access-bjbjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:10.873975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.873846    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/906d2c00-dfb6-4d0d-b6f4-ff701389d751-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.873975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.873888    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.873975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.873902    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjbjb\" (UniqueName: \"kubernetes.io/projected/906d2c00-dfb6-4d0d-b6f4-ff701389d751-kube-api-access-bjbjb\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.873975 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.873919    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/906d2c00-dfb6-4d0d-b6f4-ff701389d751-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.964855 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.964817    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:57:10.968941 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:10.968906    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-97b645-5554b95b96-fk2js"]
Sep 16 16:57:11.391629 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:11.391588    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" path="/var/lib/kubelet/pods/906d2c00-dfb6-4d0d-b6f4-ff701389d751/volumes"
Sep 16 16:57:12.621837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:12.621795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:57:14.592366 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.592299    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" podUID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" containerName="main" containerID="cri-o://b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127" gracePeriod=45
Sep 16 16:57:14.873477 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.873444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:57:14.917078 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917258 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l5l5\" (UniqueName: \"kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917258 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917165    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917258 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917285    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir\") pod \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\" (UID: \"e747ab49-4b34-4cbe-bd89-dfba2b02a381\") "
Sep 16 16:57:14.917703 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache" (OuterVolumeSpecName: "model-cache") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:14.917837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917750    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:14.917837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:14.917968 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.917863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home" (OuterVolumeSpecName: "home") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:14.919737 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.919706    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5" (OuterVolumeSpecName: "kube-api-access-9l5l5") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "kube-api-access-9l5l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:14.919953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.919918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:57:14.919953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:14.919928    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm" (OuterVolumeSpecName: "dshm") pod "e747ab49-4b34-4cbe-bd89-dfba2b02a381" (UID: "e747ab49-4b34-4cbe-bd89-dfba2b02a381"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:15.019562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.019511    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.019562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.019554    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.019562 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.019569    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l5l5\" (UniqueName: \"kubernetes.io/projected/e747ab49-4b34-4cbe-bd89-dfba2b02a381-kube-api-access-9l5l5\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.019900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.019585    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e747ab49-4b34-4cbe-bd89-dfba2b02a381-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.019900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.019598    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e747ab49-4b34-4cbe-bd89-dfba2b02a381-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.672658 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.672556    2577 generic.go:356] "Generic (PLEG): container finished" podID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" containerID="b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127" exitCode=0
Sep 16 16:57:15.672658 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.672610    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"
Sep 16 16:57:15.672658 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.672644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" event={"ID":"e747ab49-4b34-4cbe-bd89-dfba2b02a381","Type":"ContainerDied","Data":"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"}
Sep 16 16:57:15.673255 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.672692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8" event={"ID":"e747ab49-4b34-4cbe-bd89-dfba2b02a381","Type":"ContainerDied","Data":"cf6d2cac11b87688c4495f1c6de72bb89fe496e62257ace85deb87b905656bed"}
Sep 16 16:57:15.673255 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.672711    2577 scope.go:117] "RemoveContainer" containerID="b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"
Sep 16 16:57:15.681906 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.681878    2577 scope.go:117] "RemoveContainer" containerID="b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"
Sep 16 16:57:15.682251 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:15.682230    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127\": container with ID starting with b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127 not found: ID does not exist" containerID="b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"
Sep 16 16:57:15.682321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.682260    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127"} err="failed to get container status \"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127\": rpc error: code = NotFound desc = could not find container \"b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127\": container with ID starting with b994e8994dd7210584a3f5b7d1417a379279d9c217a895cf099219ce3a2cd127 not found: ID does not exist"
Sep 16 16:57:15.703335 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.703303    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:57:15.705436 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:15.705409    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-97b645/explicit-model-97b645-kserve-58db844675-sdzq8"]
Sep 16 16:57:17.392081 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:17.392045    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" path="/var/lib/kubelet/pods/e747ab49-4b34-4cbe-bd89-dfba2b02a381/volumes"
Sep 16 16:57:18.249489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:18.249449    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:57:18.249715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:18.249673    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy" containerID="cri-o://824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e" gracePeriod=30
Sep 16 16:57:19.274008 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.273971    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:19.274479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274402    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" containerName="maas-api"
Sep 16 16:57:19.274479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274415    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" containerName="maas-api"
Sep 16 16:57:19.274479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274447    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" containerName="main"
Sep 16 16:57:19.274479 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274453    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" containerName="main"
Sep 16 16:57:19.274624 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274526    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e747ab49-4b34-4cbe-bd89-dfba2b02a381" containerName="main"
Sep 16 16:57:19.274624 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.274534    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="906d2c00-dfb6-4d0d-b6f4-ff701389d751" containerName="maas-api"
Sep 16 16:57:19.277932 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.277909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.282892 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.282870    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-538267-openshift-default-dockercfg-8n5cc\""
Sep 16 16:57:19.306883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.306849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:19.368963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.368920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.368963 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.368977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ht4m\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369290 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369290 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369290 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.369498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.369433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.470842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.470800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.470842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.470842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.470887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.470938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.470965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ht4m\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471105 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471349 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471349 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471549 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.471746 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.472019 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.471996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.473752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.473731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.473922 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.473906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.483447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.483371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.483651 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.483619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ht4m\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m\") pod \"e2e-noten-538267-openshift-default-576c499686-m9z2j\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") " pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.590877 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.590836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:19.871837 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.871598    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:19.874766 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:57:19.874736    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade2c83_b5ae_4e46_a89d_5bf64648ae14.slice/crio-ec6d7cd8c7f0d2cb8bf327d285f646be3867d1906aa23c970d56c70b0d175168 WatchSource:0}: Error finding container ec6d7cd8c7f0d2cb8bf327d285f646be3867d1906aa23c970d56c70b0d175168: Status 404 returned error can't find the container with id ec6d7cd8c7f0d2cb8bf327d285f646be3867d1906aa23c970d56c70b0d175168
Sep 16 16:57:19.877076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.877039    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:19.877148 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.877118    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:19.877189 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:19.877157    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:20.704197 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:20.704165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" event={"ID":"eade2c83-b5ae-4e46-a89d-5bf64648ae14","Type":"ContainerStarted","Data":"a2be91b18ccb731514eeca6e3130b07d381792c4fadfd99a0f1cfc22614eea13"}
Sep 16 16:57:20.704657 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:20.704216    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" event={"ID":"eade2c83-b5ae-4e46-a89d-5bf64648ae14","Type":"ContainerStarted","Data":"ec6d7cd8c7f0d2cb8bf327d285f646be3867d1906aa23c970d56c70b0d175168"}
Sep 16 16:57:20.726109 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:20.726038    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" podStartSLOduration=1.726014171 podStartE2EDuration="1.726014171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:20.725621661 +0000 UTC m=+1257.811341735" watchObservedRunningTime="2026-09-16 16:57:20.726014171 +0000 UTC m=+1257.811734243"
Sep 16 16:57:21.591399 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:21.591356    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:21.597285 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:21.597252    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:21.710147 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:21.710113    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:21.711194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:21.711171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:22.909397 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:22.909331    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:57:22.909884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:22.909711    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy" containerID="cri-o://9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0" gracePeriod=30
Sep 16 16:57:23.505488 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.505455    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:57:23.619329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619441    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619492    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd7p9\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619560 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619567    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.619793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619765    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket\") pod \"29454103-f4b9-4ba1-9264-b514ec1483aa\" (UID: \"29454103-f4b9-4ba1-9264-b514ec1483aa\") "
Sep 16 16:57:23.620003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:23.620003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619922    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:23.620003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.619987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:23.620179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.620136    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.620179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.620156    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.620179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.620169    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.620412 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.620174    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:23.620896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.620640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:23.622405 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.622361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9" (OuterVolumeSpecName: "kube-api-access-zd7p9") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "kube-api-access-zd7p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:23.622531 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.622365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:23.622714 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.622696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:23.622788 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.622764    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "29454103-f4b9-4ba1-9264-b514ec1483aa" (UID: "29454103-f4b9-4ba1-9264-b514ec1483aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:23.720913 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720874    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29454103-f4b9-4ba1-9264-b514ec1483aa-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.720913 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720900    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.720913 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720910    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.720913 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720918    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.721279 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720927    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29454103-f4b9-4ba1-9264-b514ec1483aa-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.721279 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.720935    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd7p9\" (UniqueName: \"kubernetes.io/projected/29454103-f4b9-4ba1-9264-b514ec1483aa-kube-api-access-zd7p9\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:23.721934 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.721905    2577 generic.go:356] "Generic (PLEG): container finished" podID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerID="824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e" exitCode=0
Sep 16 16:57:23.722064 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.721969    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"
Sep 16 16:57:23.722064 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.721994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerDied","Data":"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"}
Sep 16 16:57:23.722064 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.722051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5" event={"ID":"29454103-f4b9-4ba1-9264-b514ec1483aa","Type":"ContainerDied","Data":"1646f63844561caeceabbab06425d5e26a971ee8379c1ef8f84048071e4c4e42"}
Sep 16 16:57:23.722181 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.722083    2577 scope.go:117] "RemoveContainer" containerID="824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"
Sep 16 16:57:23.732310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.732286    2577 scope.go:117] "RemoveContainer" containerID="5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"
Sep 16 16:57:23.741192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.741169    2577 scope.go:117] "RemoveContainer" containerID="824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"
Sep 16 16:57:23.741558 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:23.741534    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e\": container with ID starting with 824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e not found: ID does not exist" containerID="824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"
Sep 16 16:57:23.741663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.741572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e"} err="failed to get container status \"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e\": rpc error: code = NotFound desc = could not find container \"824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e\": container with ID starting with 824f323b705217b3044dceca0e9d856f15686f8d8bd893fd0923a947ebfbf27e not found: ID does not exist"
Sep 16 16:57:23.741663 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.741600    2577 scope.go:117] "RemoveContainer" containerID="5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"
Sep 16 16:57:23.741915 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:23.741891    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f\": container with ID starting with 5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f not found: ID does not exist" containerID="5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"
Sep 16 16:57:23.741957 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.741923    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f"} err="failed to get container status \"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f\": rpc error: code = NotFound desc = could not find container \"5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f\": container with ID starting with 5ccc3a77a843905597434aa7c8aa87ba3c3fca02ea9ed439c994a8b9a3bff11f not found: ID does not exist"
Sep 16 16:57:23.745472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.745444    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:57:23.747409 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.747371    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-97b645-openshift-default-74d488bbd4-k99x5"]
Sep 16 16:57:23.910915 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:23.910882    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:57:24.017219 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.017169    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" podUID="4e31d280-7ca0-44dd-a122-6914159ea17b" containerName="main" containerID="cri-o://be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0" gracePeriod=45
Sep 16 16:57:24.278597 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.278565    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:57:24.327719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.327685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.327719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.327726    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.327996 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.327749    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.327996 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.327776    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.327996 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.327922    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.328192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs5dp\" (UniqueName: \"kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp\") pod \"4e31d280-7ca0-44dd-a122-6914159ea17b\" (UID: \"4e31d280-7ca0-44dd-a122-6914159ea17b\") "
Sep 16 16:57:24.328192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328042    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home" (OuterVolumeSpecName: "home") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:24.328192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328074    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:24.328192 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328089    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:24.328445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328306    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.328445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328319    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.328445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.328328    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.330308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.330278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm" (OuterVolumeSpecName: "dshm") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:24.330458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.330336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:57:24.330458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.330354    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp" (OuterVolumeSpecName: "kube-api-access-rs5dp") pod "4e31d280-7ca0-44dd-a122-6914159ea17b" (UID: "4e31d280-7ca0-44dd-a122-6914159ea17b"). InnerVolumeSpecName "kube-api-access-rs5dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:24.429308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.429260    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e31d280-7ca0-44dd-a122-6914159ea17b-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.429308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.429298    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e31d280-7ca0-44dd-a122-6914159ea17b-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.429308 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.429309    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs5dp\" (UniqueName: \"kubernetes.io/projected/4e31d280-7ca0-44dd-a122-6914159ea17b-kube-api-access-rs5dp\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:24.730482 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.730444    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e31d280-7ca0-44dd-a122-6914159ea17b" containerID="be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0" exitCode=0
Sep 16 16:57:24.730644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.730489    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"
Sep 16 16:57:24.730644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.730518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" event={"ID":"4e31d280-7ca0-44dd-a122-6914159ea17b","Type":"ContainerDied","Data":"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"}
Sep 16 16:57:24.730644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.730555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w" event={"ID":"4e31d280-7ca0-44dd-a122-6914159ea17b","Type":"ContainerDied","Data":"87e661d3001c050fa6ed44cb623e15239f15eac638898e9c83e52bc10cb7778f"}
Sep 16 16:57:24.730644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.730573    2577 scope.go:117] "RemoveContainer" containerID="be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"
Sep 16 16:57:24.740155 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.740134    2577 scope.go:117] "RemoveContainer" containerID="be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"
Sep 16 16:57:24.740527 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:24.740504    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0\": container with ID starting with be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0 not found: ID does not exist" containerID="be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"
Sep 16 16:57:24.740591 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.740540    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0"} err="failed to get container status \"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0\": rpc error: code = NotFound desc = could not find container \"be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0\": container with ID starting with be083f7e04a1217f665602c8e14731ca82eff003ad1b267a077008a4001bb0f0 not found: ID does not exist"
Sep 16 16:57:24.751984 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.751953    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:57:24.755875 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:24.755842    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e2ea3e/test-model-e2ea3e-kserve-79f794b8cf-znx6w"]
Sep 16 16:57:25.392550 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:25.392515    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" path="/var/lib/kubelet/pods/29454103-f4b9-4ba1-9264-b514ec1483aa/volumes"
Sep 16 16:57:25.393028 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:25.393012    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e31d280-7ca0-44dd-a122-6914159ea17b" path="/var/lib/kubelet/pods/4e31d280-7ca0-44dd-a122-6914159ea17b/volumes"
Sep 16 16:57:28.172425 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.172323    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:57:28.270274 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270492 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270316    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270492 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270495    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270533    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270656 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270594    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:28.270656 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270638    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270749 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8g97\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270804 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270770    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270804 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:28.270910 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token\") pod \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\" (UID: \"73a8ee7b-91ec-4255-8ed6-3ae88d165828\") "
Sep 16 16:57:28.270974 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.270794    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:28.271056 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271036    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data" (OuterVolumeSpecName: "istio-data") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:28.271115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271069    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:28.271257 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271233    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.271257 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271257    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.271513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271270    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.271513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271282    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.271513 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.271294    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.272881 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.272846    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:28.272982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.272926    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:28.272982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.272948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97" (OuterVolumeSpecName: "kube-api-access-b8g97") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "kube-api-access-b8g97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:28.272982 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.272970    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token" (OuterVolumeSpecName: "istio-token") pod "73a8ee7b-91ec-4255-8ed6-3ae88d165828" (UID: "73a8ee7b-91ec-4255-8ed6-3ae88d165828"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:28.372204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.372161    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.372204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.372196    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8g97\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-kube-api-access-b8g97\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.372204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.372206    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.372204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.372217    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8ee7b-91ec-4255-8ed6-3ae88d165828-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:28.754928 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.754889    2577 generic.go:356] "Generic (PLEG): container finished" podID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerID="9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0" exitCode=0
Sep 16 16:57:28.755087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.754944    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"
Sep 16 16:57:28.755087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.754960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerDied","Data":"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"}
Sep 16 16:57:28.755087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.755013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr" event={"ID":"73a8ee7b-91ec-4255-8ed6-3ae88d165828","Type":"ContainerDied","Data":"ecf6108967fc8b866cfe15960d3a273d9e935d91b3e9a880594814cee0817c05"}
Sep 16 16:57:28.755087 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.755043    2577 scope.go:117] "RemoveContainer" containerID="9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"
Sep 16 16:57:28.765185 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.765165    2577 scope.go:117] "RemoveContainer" containerID="b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"
Sep 16 16:57:28.774036 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.774016    2577 scope.go:117] "RemoveContainer" containerID="9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"
Sep 16 16:57:28.774351 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:28.774330    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0\": container with ID starting with 9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0 not found: ID does not exist" containerID="9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"
Sep 16 16:57:28.774429 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.774361    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0"} err="failed to get container status \"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0\": rpc error: code = NotFound desc = could not find container \"9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0\": container with ID starting with 9ab5ea309f09a38c05bced84a6787eff3ce4b7eeb83b422088cb175c69cd20a0 not found: ID does not exist"
Sep 16 16:57:28.774429 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.774403    2577 scope.go:117] "RemoveContainer" containerID="b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"
Sep 16 16:57:28.774685 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:28.774670    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6\": container with ID starting with b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6 not found: ID does not exist" containerID="b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"
Sep 16 16:57:28.774745 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.774687    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6"} err="failed to get container status \"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6\": rpc error: code = NotFound desc = could not find container \"b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6\": container with ID starting with b970b7e428807f9cf9096dc43ee34f61ff09ac28e9d670919f8b9eafd73de7a6 not found: ID does not exist"
Sep 16 16:57:28.785045 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.785017    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:57:28.786989 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:28.786965    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f3a3973e-openshift-default-654c69f956-t6xmr"]
Sep 16 16:57:29.229734 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:29.229696    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:57:29.230221 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:29.229907    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" containerID="cri-o://e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68" gracePeriod=30
Sep 16 16:57:29.391498 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:29.391462    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" path="/var/lib/kubelet/pods/73a8ee7b-91ec-4255-8ed6-3ae88d165828/volumes"
Sep 16 16:57:32.072435 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:32.072374    2577 patch_prober.go:28] interesting pod/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:57:32.072848 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:32.072457    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:57:34.492079 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.492011    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:57:34.634843 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.634956    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv2dz\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635060 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo\") pod \"04acd5e2-add6-45d4-885a-db96de8daed9\" (UID: \"04acd5e2-add6-45d4-885a-db96de8daed9\") "
Sep 16 16:57:34.635420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635220    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:34.635420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:34.635627 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635418    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:34.635627 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:34.635627 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635500    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data" (OuterVolumeSpecName: "istio-data") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:34.635627 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635620    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.635830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635639    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.635830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635652    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04acd5e2-add6-45d4-885a-db96de8daed9-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.635830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635668    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.635830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.635680    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.637189 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.637153    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:34.637447 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.637426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz" (OuterVolumeSpecName: "kube-api-access-sv2dz") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "kube-api-access-sv2dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:34.637512 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.637452    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token" (OuterVolumeSpecName: "istio-token") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:34.637810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.637782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04acd5e2-add6-45d4-885a-db96de8daed9" (UID: "04acd5e2-add6-45d4-885a-db96de8daed9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:34.736865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.736822    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04acd5e2-add6-45d4-885a-db96de8daed9-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.736865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.736857    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04acd5e2-add6-45d4-885a-db96de8daed9-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.736865 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.736871    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.737116 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.736881    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv2dz\" (UniqueName: \"kubernetes.io/projected/04acd5e2-add6-45d4-885a-db96de8daed9-kube-api-access-sv2dz\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:34.793182 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.793144    2577 generic.go:356] "Generic (PLEG): container finished" podID="04acd5e2-add6-45d4-885a-db96de8daed9" containerID="e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68" exitCode=0
Sep 16 16:57:34.793367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.793200    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"
Sep 16 16:57:34.793367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.793232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerDied","Data":"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"}
Sep 16 16:57:34.793367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.793297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf" event={"ID":"04acd5e2-add6-45d4-885a-db96de8daed9","Type":"ContainerDied","Data":"f58035efd8722fc3b9a66dc91baf18608df6a1cea9fc3e26ed0e1d375ad10e92"}
Sep 16 16:57:34.793367 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.793327    2577 scope.go:117] "RemoveContainer" containerID="e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"
Sep 16 16:57:34.804039 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.804022    2577 scope.go:117] "RemoveContainer" containerID="e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"
Sep 16 16:57:34.812715 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.812691    2577 scope.go:117] "RemoveContainer" containerID="e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"
Sep 16 16:57:34.813080 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:34.813056    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68\": container with ID starting with e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68 not found: ID does not exist" containerID="e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"
Sep 16 16:57:34.813128 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.813105    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68"} err="failed to get container status \"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68\": rpc error: code = NotFound desc = could not find container \"e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68\": container with ID starting with e50ee8957be44e907956d80ecae2b5e0b8bffeb2c1a6a897b72bd94bba105e68 not found: ID does not exist"
Sep 16 16:57:34.813128 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.813126    2577 scope.go:117] "RemoveContainer" containerID="e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"
Sep 16 16:57:34.813428 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:57:34.813409    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890\": container with ID starting with e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890 not found: ID does not exist" containerID="e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"
Sep 16 16:57:34.813473 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.813435    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890"} err="failed to get container status \"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890\": rpc error: code = NotFound desc = could not find container \"e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890\": container with ID starting with e2741e256141e7204ae91b24b39966eecd99a9fc3c37c5c208d52e22102c0890 not found: ID does not exist"
Sep 16 16:57:34.822995 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.822959    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:57:34.830888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:34.830854    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e2ea3e-openshift-default-8b88f74c6-2h2vf"]
Sep 16 16:57:35.390733 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:35.390698    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" path="/var/lib/kubelet/pods/04acd5e2-add6-45d4-885a-db96de8daed9/volumes"
Sep 16 16:57:37.687578 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.687527    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:57:37.688007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.687974    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:37.688007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.687987    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:37.688007 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688005    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e31d280-7ca0-44dd-a122-6914159ea17b" containerName="main"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688012    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e31d280-7ca0-44dd-a122-6914159ea17b" containerName="main"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688019    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688024    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688038    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688044    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688060    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:37.688115 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688065    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688134    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688151    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a8ee7b-91ec-4255-8ed6-3ae88d165828" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688163    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688171    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:37.688315 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.688178    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e31d280-7ca0-44dd-a122-6914159ea17b" containerName="main"
Sep 16 16:57:37.691247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.691226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.694752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.694719    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-538267\"/\"noten-model-538267-kserve-self-signed-certs\""
Sep 16 16:57:37.694752 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.694741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-538267\"/\"default-dockercfg-df5pw\""
Sep 16 16:57:37.694977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.694796    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-538267\"/\"openshift-service-ca.crt\""
Sep 16 16:57:37.694977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.694806    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-538267\"/\"kube-root-ca.crt\""
Sep 16 16:57:37.700217 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.700189    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:57:37.871891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.871849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.871891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.871895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42swr\" (UniqueName: \"kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.872117 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.871990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.872117 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.872061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.872117 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.872111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.872266 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.872135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973431 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42swr\" (UniqueName: \"kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.973868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.973854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.976015 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.975975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.976252 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.976232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:37.981992 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:37.981965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42swr\" (UniqueName: \"kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr\") pod \"noten-model-538267-kserve-5d6d5bf75f-7j2jr\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") " pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:38.003198 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:38.003158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:38.282155 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:38.282129    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:57:38.284507 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:57:38.284475    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11fd1e22_7a70_455f_b4a2_cdcaf7ed54a2.slice/crio-d12ed18f4a473bea311ef03ce687ecd27b7ff8f95fe7d84088a6fdfcf63b6fd4 WatchSource:0}: Error finding container d12ed18f4a473bea311ef03ce687ecd27b7ff8f95fe7d84088a6fdfcf63b6fd4: Status 404 returned error can't find the container with id d12ed18f4a473bea311ef03ce687ecd27b7ff8f95fe7d84088a6fdfcf63b6fd4
Sep 16 16:57:38.817401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:38.817344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" event={"ID":"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2","Type":"ContainerStarted","Data":"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"}
Sep 16 16:57:38.817401 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:38.817402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" event={"ID":"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2","Type":"ContainerStarted","Data":"d12ed18f4a473bea311ef03ce687ecd27b7ff8f95fe7d84088a6fdfcf63b6fd4"}
Sep 16 16:57:38.835474 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:38.835421    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" podStartSLOduration=1.83540545 podStartE2EDuration="1.83540545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:38.834023851 +0000 UTC m=+1275.919743925" watchObservedRunningTime="2026-09-16 16:57:38.83540545 +0000 UTC m=+1275.921125516"
Sep 16 16:57:46.908668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:46.908628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:57:48.003907 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.003848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:48.004341 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.003938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:48.016398 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.016355    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:48.682977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.682941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:57:48.683194 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.683170    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy" containerID="cri-o://b37f1071da1cb3b38df8f54f98eedac2bf9cd5fd561720e2d945ce93deb07f44" gracePeriod=30
Sep 16 16:57:48.884832 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:48.884780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:57:50.662755 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:50.662717    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:57:50.782530 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:50.782489    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:50.782796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:50.782762    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerName="istio-proxy" containerID="cri-o://a2be91b18ccb731514eeca6e3130b07d381792c4fadfd99a0f1cfc22614eea13" gracePeriod=30
Sep 16 16:57:51.712567 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:51.712532    2577 patch_prober.go:28] interesting pod/e2e-noten-538267-openshift-default-576c499686-m9z2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:57:51.712974 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:51.712592    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:57:53.905371 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:53.905332    2577 generic.go:356] "Generic (PLEG): container finished" podID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerID="b37f1071da1cb3b38df8f54f98eedac2bf9cd5fd561720e2d945ce93deb07f44" exitCode=0
Sep 16 16:57:53.905810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:53.905403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerDied","Data":"b37f1071da1cb3b38df8f54f98eedac2bf9cd5fd561720e2d945ce93deb07f44"}
Sep 16 16:57:53.905810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:53.905469    2577 scope.go:117] "RemoveContainer" containerID="4dc305946256d617dd70b3c52d802a86f3160a60b27e45ac997b77e7981ff2c4"
Sep 16 16:57:53.953294 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:53.953216    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:57:54.045458 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045417    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b9xn\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045598    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045642 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045856 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045856 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.045964 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045894    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:54.046032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045957    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:54.046032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045975    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:54.046032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.046032 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.045990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:54.046206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046065    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy\") pod \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\" (UID: \"f6157255-be2b-4e13-aa49-8ee1eb8d54a7\") "
Sep 16 16:57:54.046242 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046223    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:54.046507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046487    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.046507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046507    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.046675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046518    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.046675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046526    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.046675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.046535    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.048161 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.048127    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:54.048289 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.048183    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:54.048289 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.048236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn" (OuterVolumeSpecName: "kube-api-access-6b9xn") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "kube-api-access-6b9xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:54.048289 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.048259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6157255-be2b-4e13-aa49-8ee1eb8d54a7" (UID: "f6157255-be2b-4e13-aa49-8ee1eb8d54a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:54.147998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.147950    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.147998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.147990    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b9xn\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-kube-api-access-6b9xn\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.147998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.148001    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.147998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.148009    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6157255-be2b-4e13-aa49-8ee1eb8d54a7-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:54.350540 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.350500    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 16:57:54.350973 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.350954    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:54.350973 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.350973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.350993    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351004    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351022    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351027    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29454103-f4b9-4ba1-9264-b514ec1483aa" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351098    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="04acd5e2-add6-45d4-885a-db96de8daed9" containerName="istio-proxy"
Sep 16 16:57:54.351111 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351112    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:54.351481 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.351119    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:54.353654 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.353628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.356275 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.356232    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-52fd8c-serving-cert\""
Sep 16 16:57:54.356275 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.356252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:57:54.357219 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.356241    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 16:57:54.357816 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.357795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 16:57:54.357942 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.357851    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 16:57:54.370568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.370495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 16:57:54.451941 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.451889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.452133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.451999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r75gk\" (UniqueName: \"kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.452133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.452087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.452133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.452128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.553561 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.553523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.553561 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.553571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.553798 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.553635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.553798 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.553773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r75gk\" (UniqueName: \"kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.554344 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.554315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.556159 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.556132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.556298 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.556255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.564998 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.564967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r75gk\" (UniqueName: \"kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk\") pod \"maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.682090 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.681982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:54.912924 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.912894    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"
Sep 16 16:57:54.912924 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.912898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5" event={"ID":"f6157255-be2b-4e13-aa49-8ee1eb8d54a7","Type":"ContainerDied","Data":"4dd688dbbea702795945f17a599c455d7cde04e60bff948183d257531e02b0d0"}
Sep 16 16:57:54.913476 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.912969    2577 scope.go:117] "RemoveContainer" containerID="b37f1071da1cb3b38df8f54f98eedac2bf9cd5fd561720e2d945ce93deb07f44"
Sep 16 16:57:54.938859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.938772    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:57:54.948742 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.948707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ea8f09-openshift-default-569c9cdf46-cj7l5"]
Sep 16 16:57:54.961167 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.961134    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 16:57:54.966105 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:57:54.966068    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac9100e_638f_4949_aa80_ea6167a8d3d9.slice/crio-aeeba2fd952e634c8aaa94da95f26c3e7710cee873a3729404300cad7d5dec45 WatchSource:0}: Error finding container aeeba2fd952e634c8aaa94da95f26c3e7710cee873a3729404300cad7d5dec45: Status 404 returned error can't find the container with id aeeba2fd952e634c8aaa94da95f26c3e7710cee873a3729404300cad7d5dec45
Sep 16 16:57:54.967820 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:54.967791    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:57:55.391445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.391413    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" path="/var/lib/kubelet/pods/f6157255-be2b-4e13-aa49-8ee1eb8d54a7/volumes"
Sep 16 16:57:55.923595 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.923558    2577 generic.go:356] "Generic (PLEG): container finished" podID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerID="a2be91b18ccb731514eeca6e3130b07d381792c4fadfd99a0f1cfc22614eea13" exitCode=0
Sep 16 16:57:55.924091 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.923630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" event={"ID":"eade2c83-b5ae-4e46-a89d-5bf64648ae14","Type":"ContainerDied","Data":"a2be91b18ccb731514eeca6e3130b07d381792c4fadfd99a0f1cfc22614eea13"}
Sep 16 16:57:55.925587 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.925552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" event={"ID":"0ac9100e-638f-4949-aa80-ea6167a8d3d9","Type":"ContainerStarted","Data":"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"}
Sep 16 16:57:55.925751 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.925624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" event={"ID":"0ac9100e-638f-4949-aa80-ea6167a8d3d9","Type":"ContainerStarted","Data":"aeeba2fd952e634c8aaa94da95f26c3e7710cee873a3729404300cad7d5dec45"}
Sep 16 16:57:55.925751 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.925666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:57:55.945969 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:55.945910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" podStartSLOduration=1.644856833 podStartE2EDuration="1.9458894s" totalImagesPullingTime="301.032567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:55.94348395 +0000 UTC m=+1293.029204025" watchObservedRunningTime="2026-09-16 16:57:55.9458894 +0000 UTC m=+1293.031609473"
Sep 16 16:57:56.049566 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.049488    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:56.171282 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171478 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171339    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171478 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171449    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171565 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ht4m\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171602 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171647 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171606    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171699 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket\") pod \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\" (UID: \"eade2c83-b5ae-4e46-a89d-5bf64648ae14\") "
Sep 16 16:57:56.171853 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:56.172011 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data" (OuterVolumeSpecName: "istio-data") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:56.172011 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.171990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:56.172135 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172004    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:56.172135 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172104    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:56.172135 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172127    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.172283 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172144    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.172283 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172157    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.172283 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.172170    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.173697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.173660    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:56.173900 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.173881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:56.174106 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.174085    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token" (OuterVolumeSpecName: "istio-token") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:56.174166 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.174099    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m" (OuterVolumeSpecName: "kube-api-access-8ht4m") pod "eade2c83-b5ae-4e46-a89d-5bf64648ae14" (UID: "eade2c83-b5ae-4e46-a89d-5bf64648ae14"). InnerVolumeSpecName "kube-api-access-8ht4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:56.273183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.273137    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.273183 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.273181    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ht4m\" (UniqueName: \"kubernetes.io/projected/eade2c83-b5ae-4e46-a89d-5bf64648ae14-kube-api-access-8ht4m\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.273533 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.273196    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.273533 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.273210    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.273533 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.273223    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eade2c83-b5ae-4e46-a89d-5bf64648ae14-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:57:56.325248 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325143    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 16:57:56.325783 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325762    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerName="istio-proxy"
Sep 16 16:57:56.325869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325787    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerName="istio-proxy"
Sep 16 16:57:56.325869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325804    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:56.325869 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325813    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6157255-be2b-4e13-aa49-8ee1eb8d54a7" containerName="istio-proxy"
Sep 16 16:57:56.326034 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.325932    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" containerName="istio-proxy"
Sep 16 16:57:56.329644 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.329618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.332107 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.332074    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-2ce8bc-openshift-default-dockercfg-gsn9v\""
Sep 16 16:57:56.343691 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.343653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 16:57:56.475223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475223 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475650 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475650 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.475650 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.475584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5slvb\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576684 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576684 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5slvb\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.576940 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.576825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577151 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577268 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577268 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577268 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577697 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.577815 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.577796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.579219 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.579194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.579853 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.579830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.585606 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.585571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.585769 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.585617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5slvb\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb\") pod \"e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") " pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.642421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.642334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:56.924656 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.924623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 16:57:56.927663 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:57:56.927626    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ad5601_3f8b_4e88_8aec_c827863223cd.slice/crio-a0c6e05d1eff63a7d3acbf717b0e3b0bdb71d7ab75fb6bd665f3aec5af669ec9 WatchSource:0}: Error finding container a0c6e05d1eff63a7d3acbf717b0e3b0bdb71d7ab75fb6bd665f3aec5af669ec9: Status 404 returned error can't find the container with id a0c6e05d1eff63a7d3acbf717b0e3b0bdb71d7ab75fb6bd665f3aec5af669ec9
Sep 16 16:57:56.930744 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.930705    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:56.930866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.930787    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:56.930866 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.930819    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:57:56.933844 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.933820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"
Sep 16 16:57:56.933968 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.933848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j" event={"ID":"eade2c83-b5ae-4e46-a89d-5bf64648ae14","Type":"ContainerDied","Data":"ec6d7cd8c7f0d2cb8bf327d285f646be3867d1906aa23c970d56c70b0d175168"}
Sep 16 16:57:56.933968 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.933910    2577 scope.go:117] "RemoveContainer" containerID="a2be91b18ccb731514eeca6e3130b07d381792c4fadfd99a0f1cfc22614eea13"
Sep 16 16:57:56.957465 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.957432    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:56.961076 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:56.961046    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-538267-openshift-default-576c499686-m9z2j"]
Sep 16 16:57:57.392038 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:57.391995    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eade2c83-b5ae-4e46-a89d-5bf64648ae14" path="/var/lib/kubelet/pods/eade2c83-b5ae-4e46-a89d-5bf64648ae14/volumes"
Sep 16 16:57:57.942034 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:57.941996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerStarted","Data":"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"}
Sep 16 16:57:57.942540 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:57.942090    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerStarted","Data":"a0c6e05d1eff63a7d3acbf717b0e3b0bdb71d7ab75fb6bd665f3aec5af669ec9"}
Sep 16 16:57:57.998376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:57.998311    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podStartSLOduration=1.998290917 podStartE2EDuration="1.998290917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:57.996372434 +0000 UTC m=+1295.082092507" watchObservedRunningTime="2026-09-16 16:57:57.998290917 +0000 UTC m=+1295.084011280"
Sep 16 16:57:58.642773 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:58.642728    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:58.648450 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:58.648418    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:58.948745 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:58.948652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:57:58.949576 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:57:58.949558    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:58:01.659441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:01.659397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:58:01.942228 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:01.942147    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 16:58:05.910236 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:05.910169    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" podUID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" containerName="main" containerID="cri-o://45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c" gracePeriod=45
Sep 16 16:58:06.176831 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.176751    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:58:06.277506 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277472    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277707 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277560    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277707 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277707 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42swr\" (UniqueName: \"kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277707 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277684    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277927 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home\") pod \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\" (UID: \"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2\") "
Sep 16 16:58:06.277927 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:58:06.278038 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.277992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home" (OuterVolumeSpecName: "home") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:58:06.278092 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.278078    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.278150 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.278094    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.278238 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.278212    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:58:06.279947 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.279913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm" (OuterVolumeSpecName: "dshm") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:58:06.280065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.280013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:58:06.280065 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.280051    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr" (OuterVolumeSpecName: "kube-api-access-42swr") pod "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" (UID: "11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2"). InnerVolumeSpecName "kube-api-access-42swr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:58:06.378795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.378747    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.378795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.378785    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.378795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.378796    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.378795 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.378806    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42swr\" (UniqueName: \"kubernetes.io/projected/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2-kube-api-access-42swr\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 16:58:06.993839 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.993798    2577 generic.go:356] "Generic (PLEG): container finished" podID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" containerID="45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c" exitCode=0
Sep 16 16:58:06.994300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.993846    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"
Sep 16 16:58:06.994300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.993862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" event={"ID":"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2","Type":"ContainerDied","Data":"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"}
Sep 16 16:58:06.994300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.993901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr" event={"ID":"11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2","Type":"ContainerDied","Data":"d12ed18f4a473bea311ef03ce687ecd27b7ff8f95fe7d84088a6fdfcf63b6fd4"}
Sep 16 16:58:06.994300 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:06.993926    2577 scope.go:117] "RemoveContainer" containerID="45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"
Sep 16 16:58:07.003883 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:07.003864    2577 scope.go:117] "RemoveContainer" containerID="45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"
Sep 16 16:58:07.004249 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:07.004227    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c\": container with ID starting with 45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c not found: ID does not exist" containerID="45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"
Sep 16 16:58:07.004302 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:07.004259    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c"} err="failed to get container status \"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c\": rpc error: code = NotFound desc = could not find container \"45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c\": container with ID starting with 45bc56a827f10f96df995bc8af0482afa241c886842560c67a9b9d7dd441377c not found: ID does not exist"
Sep 16 16:58:07.015625 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:07.015578    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:58:07.022001 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:07.021972    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-538267/noten-model-538267-kserve-5d6d5bf75f-7j2jr"]
Sep 16 16:58:07.391920 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:07.391883    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" path="/var/lib/kubelet/pods/11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2/volumes"
Sep 16 16:58:08.403120 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.403078    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 16:58:08.403564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.403507    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" containerName="main"
Sep 16 16:58:08.403564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.403521    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" containerName="main"
Sep 16 16:58:08.403648 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.403584    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11fd1e22-7a70-455f-b4a2-cdcaf7ed54a2" containerName="main"
Sep 16 16:58:08.408297 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.408272    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.410613 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.410590    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-34d301-openshift-default-dockercfg-hzrrc\""
Sep 16 16:58:08.432743 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.431951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 16:58:08.602368 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602643 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djlm6\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602763 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.602763 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.602694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703588 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703588 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djlm6\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703796 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703907 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.703955 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704011 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.703987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704053 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704133 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704189 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704262 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704334 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.704507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.704488    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.706215 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.706193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.706492 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.706472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.722179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.722149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:08.723953 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:08.723927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djlm6\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6\") pod \"e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") " pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:09.021096 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:09.021005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:09.299668 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:09.299610    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 16:58:09.301979 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:58:09.301953    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf46f4a_5d20_4b97_b45f_300886bd5efd.slice/crio-314eab65f8f2529e9dda27bf5fb040dc6dae0375eb8a8740ed72f60121684f26 WatchSource:0}: Error finding container 314eab65f8f2529e9dda27bf5fb040dc6dae0375eb8a8740ed72f60121684f26: Status 404 returned error can't find the container with id 314eab65f8f2529e9dda27bf5fb040dc6dae0375eb8a8740ed72f60121684f26
Sep 16 16:58:09.304253 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:09.304216    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:09.304358 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:09.304306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:09.304358 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:09.304352    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:10.013997 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:10.013960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerStarted","Data":"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"}
Sep 16 16:58:10.014523 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:10.014017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerStarted","Data":"314eab65f8f2529e9dda27bf5fb040dc6dae0375eb8a8740ed72f60121684f26"}
Sep 16 16:58:10.021420 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:10.021364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:10.026646 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:10.026618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:10.033867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:10.033807    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podStartSLOduration=2.033789066 podStartE2EDuration="2.033789066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:10.031858417 +0000 UTC m=+1307.117578481" watchObservedRunningTime="2026-09-16 16:58:10.033789066 +0000 UTC m=+1307.119509137"
Sep 16 16:58:11.020104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:11.020071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:11.021149 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:11.021120    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:15.111232 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.111195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 16:58:15.115359 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.115331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.118891 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.118845    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-34d301-serving-cert\""
Sep 16 16:58:15.131711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.131667    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 16:58:15.172581 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.172540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.172780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.172690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.172780 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.172758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.172868 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.172801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mx9\" (UniqueName: \"kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.273801 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.273760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.273992 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.273833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mx9\" (UniqueName: \"kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.273992 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.273877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.273992 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.273968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.274122 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:15.274099    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-serving-cert: secret "maas-api-e2e-shared-b-w3-34d301-serving-cert" not found
Sep 16 16:58:15.274181 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:15.274170    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls podName:d4f6da88-3dc3-4793-8fdb-b33a73d05309 nodeName:}" failed. No retries permitted until 2026-09-16 16:58:15.774148815 +0000 UTC m=+1312.859868883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls") pod "maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" (UID: "d4f6da88-3dc3-4793-8fdb-b33a73d05309") : secret "maas-api-e2e-shared-b-w3-34d301-serving-cert" not found
Sep 16 16:58:15.274682 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.274659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.276415 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.276377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.285860 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.285836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mx9\" (UniqueName: \"kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.780109 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.780069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:15.782719 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:15.782693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:16.033399 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:16.033005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:16.322548 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:16.322506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 16:58:17.068329 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:17.068291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" event={"ID":"d4f6da88-3dc3-4793-8fdb-b33a73d05309","Type":"ContainerStarted","Data":"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"}
Sep 16 16:58:17.068577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:17.068355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" event={"ID":"d4f6da88-3dc3-4793-8fdb-b33a73d05309","Type":"ContainerStarted","Data":"abc83cca99bd2497c33131a2873fd1d6e3aad1218ddbc531aad67ca6132fa1a9"}
Sep 16 16:58:17.068577 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:17.068431    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:17.104310 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:17.104249    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" podStartSLOduration=1.802319937 podStartE2EDuration="2.104230104s" totalImagesPullingTime="301.910167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:17.102596937 +0000 UTC m=+1314.188317012" watchObservedRunningTime="2026-09-16 16:58:17.104230104 +0000 UTC m=+1314.189950237"
Sep 16 16:58:19.515287 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:19.515244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:58:22.102039 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.102005    2577 generic.go:356] "Generic (PLEG): container finished" podID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerID="29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba" exitCode=0
Sep 16 16:58:22.102517 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.102078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerDied","Data":"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"}
Sep 16 16:58:22.102569 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.102540    2577 scope.go:117] "RemoveContainer" containerID="29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"
Sep 16 16:58:22.103744 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.103714    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:22.103820 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.103786    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:22.103820 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:22.103815    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:23.084791 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:23.084761    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 16:58:23.109893 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:23.109864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerStarted","Data":"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"}
Sep 16 16:58:23.643503 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:23.643374    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:58:23.646580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:23.646534    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 16 16:58:23.646765 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:23.646609    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 16 16:58:24.642836 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:24.642796    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 16 16:58:24.643483 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:24.643448    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 16 16:58:25.642977 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:25.642927    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 16 16:58:25.643555 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:25.643003    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 16 16:58:26.643475 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:26.643353    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 16 16:58:26.644492 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:26.644039    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 16 16:58:27.648415 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:27.647517    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 16 16:58:27.648415 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:27.647585    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 16 16:58:28.647842 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:28.647807    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:58:28.648165 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:28.648135    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:58:28.648896 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:28.648877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 16:58:29.154020 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.153985    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 16:58:29.157454 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.157422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.161179 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.161149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6148d487-openshift-default-dockercfg-zvjwh\""
Sep 16 16:58:29.172585 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.172555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 16:58:29.258859 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.258818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259117 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt4n7\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259279 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259421 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259545 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259678 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259817 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.259961 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.259921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361285 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361285 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361579 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt4n7\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361888 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361762    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.361994 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.361909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.362043 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.362023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.362311 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.362281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.362489 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.362286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.364072 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.364048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.364229 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.364206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.374242 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.374198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt4n7\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.376079 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.376050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token\") pod \"e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") " pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.473630 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.473539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:29.759365 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.759278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 16:58:29.762694 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:58:29.762657    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db8d2d4_9aac_4d71_8247_d92e4be3d555.slice/crio-5769430d4a08842fba89e907944421009e707175e55c6c60807be5a765ca6422 WatchSource:0}: Error finding container 5769430d4a08842fba89e907944421009e707175e55c6c60807be5a765ca6422: Status 404 returned error can't find the container with id 5769430d4a08842fba89e907944421009e707175e55c6c60807be5a765ca6422
Sep 16 16:58:29.765241 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.765204    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:29.765319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.765277    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:29.765319 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:29.765311    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:30.178049 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.177950    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerID="6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc" exitCode=0
Sep 16 16:58:30.178049 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.178028    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerDied","Data":"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"}
Sep 16 16:58:30.178540 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.178520    2577 scope.go:117] "RemoveContainer" containerID="6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"
Sep 16 16:58:30.179814 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.179780    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:30.179934 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.179876    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:30.180003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.179932    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:30.180003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.179945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerStarted","Data":"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"}
Sep 16 16:58:30.180003 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.179985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerStarted","Data":"5769430d4a08842fba89e907944421009e707175e55c6c60807be5a765ca6422"}
Sep 16 16:58:30.221852 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.221282    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podStartSLOduration=1.221257027 podStartE2EDuration="1.221257027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:30.218112695 +0000 UTC m=+1327.303832768" watchObservedRunningTime="2026-09-16 16:58:30.221257027 +0000 UTC m=+1327.306977100"
Sep 16 16:58:30.474303 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.474202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:30.480553 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:30.480522    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:31.187803 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:31.187769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerStarted","Data":"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"}
Sep 16 16:58:31.188251 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:31.188103    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:31.188937 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:31.188915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:32.021484 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:32.021441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:32.021828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:32.021797    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 16 16:58:32.022057 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:32.022029    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 16 16:58:33.022376 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:33.022326    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 16 16:58:33.022937 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:33.022435    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 16 16:58:34.023884 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:34.023844    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 16 16:58:34.024527 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:34.024489    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 16 16:58:35.022323 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:35.022282    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 16 16:58:35.023100 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:35.022646    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 16 16:58:36.022242 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.022200    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 16 16:58:36.022886 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.022850    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 16 16:58:36.172304 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.172265    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 16:58:36.175810 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.175781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.178247 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.178215    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6148d487-serving-cert\""
Sep 16 16:58:36.185441 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.185415    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 16:58:36.338903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.338858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.338903 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.338904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.339134 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.338941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsr6d\" (UniqueName: \"kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.339134 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.338996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.440675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.440634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.440675 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.440675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.440966 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.440697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsr6d\" (UniqueName: \"kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.440966 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.440715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.440966 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:36.440848    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-serving-cert: secret "maas-api-e2e-mt-6148d487-serving-cert" not found
Sep 16 16:58:36.440966 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:36.440939    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls podName:d5d62249-4aa0-40f3-bca2-3f89c12845ab nodeName:}" failed. No retries permitted until 2026-09-16 16:58:36.940916171 +0000 UTC m=+1334.026636246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls") pod "maas-api-e2e-mt-6148d487-64d98f84-r55sn" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab") : secret "maas-api-e2e-mt-6148d487-serving-cert" not found
Sep 16 16:58:36.441472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.441450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.443260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.443238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.449727 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.449681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsr6d\" (UniqueName: \"kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.946324 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:36.946290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:36.946532 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:36.946484    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-serving-cert: secret "maas-api-e2e-mt-6148d487-serving-cert" not found
Sep 16 16:58:36.946579 ip-10-0-132-17 kubenswrapper[2577]: E0916 16:58:36.946547    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls podName:d5d62249-4aa0-40f3-bca2-3f89c12845ab nodeName:}" failed. No retries permitted until 2026-09-16 16:58:37.946529959 +0000 UTC m=+1335.032250028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls") pod "maas-api-e2e-mt-6148d487-64d98f84-r55sn" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab") : secret "maas-api-e2e-mt-6148d487-serving-cert" not found
Sep 16 16:58:37.038840 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.038806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:37.039516 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.039437    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:37.040414 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.040373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 16:58:37.957909 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.957864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:37.960515 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.960478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"maas-api-e2e-mt-6148d487-64d98f84-r55sn\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:37.987867 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:37.987829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:38.278877 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:38.278847    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 16:58:38.281112 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:58:38.281081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d62249_4aa0_40f3_bca2_3f89c12845ab.slice/crio-3269bb5695c4e4e73ed70107e74cee40e23c76db3b632c082f7fac4252e1076e WatchSource:0}: Error finding container 3269bb5695c4e4e73ed70107e74cee40e23c76db3b632c082f7fac4252e1076e: Status 404 returned error can't find the container with id 3269bb5695c4e4e73ed70107e74cee40e23c76db3b632c082f7fac4252e1076e
Sep 16 16:58:39.234173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:39.234069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" event={"ID":"d5d62249-4aa0-40f3-bca2-3f89c12845ab","Type":"ContainerStarted","Data":"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"}
Sep 16 16:58:39.234173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:39.234141    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:39.234173 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:39.234162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" event={"ID":"d5d62249-4aa0-40f3-bca2-3f89c12845ab","Type":"ContainerStarted","Data":"3269bb5695c4e4e73ed70107e74cee40e23c76db3b632c082f7fac4252e1076e"}
Sep 16 16:58:39.255013 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:39.254945    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" podStartSLOduration=2.68865453 podStartE2EDuration="3.25492669s" totalImagesPullingTime="566.27216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:39.252318211 +0000 UTC m=+1336.338038283" watchObservedRunningTime="2026-09-16 16:58:39.25492669 +0000 UTC m=+1336.340646790"
Sep 16 16:58:40.816321 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:40.816264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:58:45.248373 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.248338    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 16:58:45.513280 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.513192    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 16:58:45.516023 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.515997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.519229 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.519170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"auth-test-model-52fd8c-kserve-self-signed-certs\""
Sep 16 16:58:45.519445 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.519255    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"kube-root-ca.crt\""
Sep 16 16:58:45.537097 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.537063    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"openshift-service-ca.crt\""
Sep 16 16:58:45.539045 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.537432    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"default-dockercfg-hsq9j\""
Sep 16 16:58:45.546598 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.546563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6rcs\" (UniqueName: \"kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.643411 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.643277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744035 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.743969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744035 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6rcs\" (UniqueName: \"kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744326 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.744970 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.744840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.745036 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.745019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.746782 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.746757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.747026 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.747004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.753830 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.753804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6rcs\" (UniqueName: \"kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs\") pod \"auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:45.852261 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:45.852143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:46.132068 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:46.131972    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 16:58:46.275798 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:46.275756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" event={"ID":"936ddc66-9bba-4a96-9470-889c5258e82b","Type":"ContainerStarted","Data":"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"}
Sep 16 16:58:46.275798 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:46.275801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" event={"ID":"936ddc66-9bba-4a96-9470-889c5258e82b","Type":"ContainerStarted","Data":"2c128b00ffb568dc7b35cc57f40080c32c8d616a1fad4a57d8dc324af6d66dfc"}
Sep 16 16:58:46.298104 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:46.298046    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" podStartSLOduration=1.2980264049999999 podStartE2EDuration="1.298026405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:46.29727088 +0000 UTC m=+1343.382990953" watchObservedRunningTime="2026-09-16 16:58:46.298026405 +0000 UTC m=+1343.383746476"
Sep 16 16:58:51.311886 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.311847    2577 generic.go:356] "Generic (PLEG): container finished" podID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerID="83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc" exitCode=0
Sep 16 16:58:51.312472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.311920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerDied","Data":"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"}
Sep 16 16:58:51.312472 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.312346    2577 scope.go:117] "RemoveContainer" containerID="83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"
Sep 16 16:58:51.313568 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.313536    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:51.313634 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.313609    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:51.313676 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.313642    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 16:58:51.418338 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:51.418296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:58:52.321450 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:52.320365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerStarted","Data":"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"}
Sep 16 16:58:52.474204 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:52.474154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:52.474449 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:52.474425    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:52.474507 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:52.474487    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:53.474544 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:53.474496    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:53.475049 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:53.474564    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:54.474652 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:54.474610    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:54.475085 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:54.474680    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:55.473978 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:55.473931    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:55.474193 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:55.473999    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:55.852793 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:55.852741    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:55.853255 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:55.852813    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:55.865711 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:55.865678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:56.363075 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:56.363043    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 16:58:56.474154 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:56.474118    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:56.474338 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:56.474181    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:57.474284 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:57.474243    2577 patch_prober.go:28] interesting pod/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 16 16:58:57.474828 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:57.474314    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 16 16:58:58.478555 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:58.478516    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:58.479017 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:58.478805    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:58:58.479688 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:58:58.479664    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 16:59:12.885680 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.885637    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 16:59:12.889233 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.889205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:12.892336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.892303    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"default-dockercfg-vskgs\""
Sep 16 16:59:12.892336 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.892341    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"kube-root-ca.crt\""
Sep 16 16:59:12.892590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.892313    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"openshift-service-ca.crt\""
Sep 16 16:59:12.892590 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.892467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"auth-test-model-34d301-kserve-self-signed-certs\""
Sep 16 16:59:12.898956 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:12.898928    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 16:59:13.026504 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.026504 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.026730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.026730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.026730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr8rl\" (UniqueName: \"kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.026730 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.026701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr8rl\" (UniqueName: \"kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128580 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128802 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128858 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.128858 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.128831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.130801 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.130777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.130921 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.130861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.136757 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.136698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr8rl\" (UniqueName: \"kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl\") pod \"auth-test-model-34d301-kserve-86b8df49bc-2xdt8\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") " pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.200142 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.200105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:13.476355 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:13.476228    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 16:59:13.479501 ip-10-0-132-17 kubenswrapper[2577]: W0916 16:59:13.479455    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67e5198f_d25f_4ee9_9106_696ad6173c0a.slice/crio-46a832448d08eb7eec9b631bce392d3dc4d1f8eaf220ee9c32e2f446bfb9eb94 WatchSource:0}: Error finding container 46a832448d08eb7eec9b631bce392d3dc4d1f8eaf220ee9c32e2f446bfb9eb94: Status 404 returned error can't find the container with id 46a832448d08eb7eec9b631bce392d3dc4d1f8eaf220ee9c32e2f446bfb9eb94
Sep 16 16:59:14.458260 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:14.458223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" event={"ID":"67e5198f-d25f-4ee9-9106-696ad6173c0a","Type":"ContainerStarted","Data":"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"}
Sep 16 16:59:14.458777 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:14.458274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" event={"ID":"67e5198f-d25f-4ee9-9106-696ad6173c0a","Type":"ContainerStarted","Data":"46a832448d08eb7eec9b631bce392d3dc4d1f8eaf220ee9c32e2f446bfb9eb94"}
Sep 16 16:59:14.476206 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:14.476141    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" podStartSLOduration=2.476121784 podStartE2EDuration="2.476121784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:59:14.474999058 +0000 UTC m=+1371.560719130" watchObservedRunningTime="2026-09-16 16:59:14.476121784 +0000 UTC m=+1371.561841857"
Sep 16 16:59:17.703564 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:17.703523    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 16:59:23.201198 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:23.201154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:23.201761 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:23.201218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:23.214159 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:23.214121    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:23.520290 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:23.520205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 16:59:47.604271 ip-10-0-132-17 kubenswrapper[2577]: I0916 16:59:47.604231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:00:00.148131 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.148086    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"]
Sep 16 17:00:00.150737 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.150717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"
Sep 16 17:00:00.152954 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.152926    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:00:00.162953 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.162922    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"
Sep 16 17:00:00.169688 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.169616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"]
Sep 16 17:00:00.444271 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.444243    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"]
Sep 16 17:00:00.446490 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:00:00.446459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8ac18d_681d_44c1_9dd3_d9be8f9cc408.slice/crio-114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e WatchSource:0}: Error finding container 114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e: Status 404 returned error can't find the container with id 114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e
Sep 16 17:00:00.725205 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:00.725169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b" event={"ID":"eb8ac18d-681d-44c1-9dd3-d9be8f9cc408","Type":"ContainerStarted","Data":"114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e"}
Sep 16 17:00:01.707759 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:01.707663    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:00:01.731897 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:01.731855    2577 generic.go:356] "Generic (PLEG): container finished" podID="eb8ac18d-681d-44c1-9dd3-d9be8f9cc408" containerID="ed9f28683db20591cfd1a70fbd8ff2cfe0fb69314f4ba9573abc41b202c034ee" exitCode=0
Sep 16 17:00:01.732084 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:01.731913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b" event={"ID":"eb8ac18d-681d-44c1-9dd3-d9be8f9cc408","Type":"ContainerDied","Data":"ed9f28683db20591cfd1a70fbd8ff2cfe0fb69314f4ba9573abc41b202c034ee"}
Sep 16 17:00:02.878822 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:02.878790    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"
Sep 16 17:00:03.745285 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:03.745201    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b"
Sep 16 17:00:03.745285 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:03.745244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826300-hnw5b" event={"ID":"eb8ac18d-681d-44c1-9dd3-d9be8f9cc408","Type":"ContainerDied","Data":"114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e"}
Sep 16 17:00:03.745503 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:03.745317    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="114324b710b347b4f8002a249c52ed9b227da3f2f2841213406e51e35ff2be7e"
Sep 16 17:00:15.505480 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:15.505441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:00:23.608866 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:23.608771    2577 scope.go:117] "RemoveContainer" containerID="e5f530be8de060a616b772700346a70542b37f6ed7eacdf54be3e38f87671f1c"
Sep 16 17:00:34.008078 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:34.008041    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:00:38.288262 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.280481    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 17:00:38.288262 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.280885    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" podUID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" containerName="maas-api" containerID="cri-o://42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3" gracePeriod=30
Sep 16 17:00:38.559426 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.559368    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 17:00:38.724800 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.724753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsr6d\" (UniqueName: \"kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d\") pod \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") "
Sep 16 17:00:38.724800 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.724808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs\") pod \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") "
Sep 16 17:00:38.725091 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.724923    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") pod \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") "
Sep 16 17:00:38.725091 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.724971    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca\") pod \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\" (UID: \"d5d62249-4aa0-40f3-bca2-3f89c12845ab\") "
Sep 16 17:00:38.725736 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.725686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d5d62249-4aa0-40f3-bca2-3f89c12845ab" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:00:38.727292 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.727243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d5d62249-4aa0-40f3-bca2-3f89c12845ab" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:00:38.728085 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.728052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d" (OuterVolumeSpecName: "kube-api-access-tsr6d") pod "d5d62249-4aa0-40f3-bca2-3f89c12845ab" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab"). InnerVolumeSpecName "kube-api-access-tsr6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:00:38.728199 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.728160    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d5d62249-4aa0-40f3-bca2-3f89c12845ab" (UID: "d5d62249-4aa0-40f3-bca2-3f89c12845ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:00:38.826556 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.826468    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.826556 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.826498    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d62249-4aa0-40f3-bca2-3f89c12845ab-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.826556 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.826508    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsr6d\" (UniqueName: \"kubernetes.io/projected/d5d62249-4aa0-40f3-bca2-3f89c12845ab-kube-api-access-tsr6d\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.826556 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.826517    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d62249-4aa0-40f3-bca2-3f89c12845ab-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.943086 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.943050    2577 generic.go:356] "Generic (PLEG): container finished" podID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" containerID="42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3" exitCode=0
Sep 16 17:00:38.943297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.943104    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"
Sep 16 17:00:38.943297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.943137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" event={"ID":"d5d62249-4aa0-40f3-bca2-3f89c12845ab","Type":"ContainerDied","Data":"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"}
Sep 16 17:00:38.943297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.943195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn" event={"ID":"d5d62249-4aa0-40f3-bca2-3f89c12845ab","Type":"ContainerDied","Data":"3269bb5695c4e4e73ed70107e74cee40e23c76db3b632c082f7fac4252e1076e"}
Sep 16 17:00:38.943297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.943219    2577 scope.go:117] "RemoveContainer" containerID="42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"
Sep 16 17:00:38.953749 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.953728    2577 scope.go:117] "RemoveContainer" containerID="42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"
Sep 16 17:00:38.954138 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:00:38.954117    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3\": container with ID starting with 42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3 not found: ID does not exist" containerID="42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"
Sep 16 17:00:38.954187 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.954148    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3"} err="failed to get container status \"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3\": rpc error: code = NotFound desc = could not find container \"42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3\": container with ID starting with 42f71430748a385c445b2bd77bc52c64b07830b60ec62107ddb2410a744da6f3 not found: ID does not exist"
Sep 16 17:00:38.967431 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.967394    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 17:00:38.973407 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:38.973352    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6148d487-64d98f84-r55sn"]
Sep 16 17:00:39.392070 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:39.392035    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" path="/var/lib/kubelet/pods/d5d62249-4aa0-40f3-bca2-3f89c12845ab/volumes"
Sep 16 17:00:52.623941 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:00:52.623903    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:01:09.901584 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:09.901545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:01:23.407088 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:23.407056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:01:23.632542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:23.632510    2577 scope.go:117] "RemoveContainer" containerID="6f268699d23084ff4ccdbac1c459ed2685251bbc3a7ac6d0cdb50550ca3bc0c5"
Sep 16 17:01:29.580631 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:29.580591    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 17:01:29.581122 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:29.580896    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy" containerID="cri-o://20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2" gracePeriod=30
Sep 16 17:01:34.880913 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:34.880788    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 17:01:35.023211 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023172    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023303    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023559 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023559 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023517    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023556    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:35.023770 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023661    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023770 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023700    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt4n7\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.023891 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.023773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert\") pod \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\" (UID: \"1db8d2d4-9aac-4d71-8247-d92e4be3d555\") "
Sep 16 17:01:35.024195 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.024139    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:35.024195 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.024158    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.024430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.024369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data" (OuterVolumeSpecName: "istio-data") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:35.024508 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.024470    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:01:35.024720 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.024695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:35.027073 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.027015    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:01:35.027320 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.027288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7" (OuterVolumeSpecName: "kube-api-access-mt4n7") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "kube-api-access-mt4n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:01:35.027777 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.027749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token" (OuterVolumeSpecName: "istio-token") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:01:35.027777 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.027759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1db8d2d4-9aac-4d71-8247-d92e4be3d555" (UID: "1db8d2d4-9aac-4d71-8247-d92e4be3d555"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125708    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125752    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125765    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125777    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125791    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.125804 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125804    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db8d2d4-9aac-4d71-8247-d92e4be3d555-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.126173 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125818    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt4n7\" (UniqueName: \"kubernetes.io/projected/1db8d2d4-9aac-4d71-8247-d92e4be3d555-kube-api-access-mt4n7\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.126173 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.125832    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db8d2d4-9aac-4d71-8247-d92e4be3d555-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:35.253763 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.253717    2577 generic.go:356] "Generic (PLEG): container finished" podID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerID="20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2" exitCode=0
Sep 16 17:01:35.253970 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.253809    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"
Sep 16 17:01:35.253970 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.253802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerDied","Data":"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"}
Sep 16 17:01:35.254098 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.253992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl" event={"ID":"1db8d2d4-9aac-4d71-8247-d92e4be3d555","Type":"ContainerDied","Data":"5769430d4a08842fba89e907944421009e707175e55c6c60807be5a765ca6422"}
Sep 16 17:01:35.254098 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.254030    2577 scope.go:117] "RemoveContainer" containerID="20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"
Sep 16 17:01:35.265512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.265450    2577 scope.go:117] "RemoveContainer" containerID="83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"
Sep 16 17:01:35.277161 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.277135    2577 scope.go:117] "RemoveContainer" containerID="20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"
Sep 16 17:01:35.277637 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:01:35.277604    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2\": container with ID starting with 20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2 not found: ID does not exist" containerID="20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"
Sep 16 17:01:35.277747 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.277652    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2"} err="failed to get container status \"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2\": rpc error: code = NotFound desc = could not find container \"20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2\": container with ID starting with 20384651619c1e5909fccbd39278427263ed85f3780c6347b71d4ee0a0b69eb2 not found: ID does not exist"
Sep 16 17:01:35.277747 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.277682    2577 scope.go:117] "RemoveContainer" containerID="83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"
Sep 16 17:01:35.278551 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:01:35.278241    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc\": container with ID starting with 83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc not found: ID does not exist" containerID="83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"
Sep 16 17:01:35.278551 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.278281    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc"} err="failed to get container status \"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc\": rpc error: code = NotFound desc = could not find container \"83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc\": container with ID starting with 83a3a39224462ac8e7c55d3f48fdb8b8c195df85b089540844ae7df59e5922bc not found: ID does not exist"
Sep 16 17:01:35.281077 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.281048    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 17:01:35.284901 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.284866    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6148d487-openshift-default-5b86bb995b-mn6rl"]
Sep 16 17:01:35.391790 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:35.391705    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" path="/var/lib/kubelet/pods/1db8d2d4-9aac-4d71-8247-d92e4be3d555/volumes"
Sep 16 17:01:39.815410 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:39.813819    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 17:01:40.764730 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:40.764693    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 17:01:43.802554 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:43.802519    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:01:54.840782 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:54.840713    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" podUID="936ddc66-9bba-4a96-9470-889c5258e82b" containerName="main" containerID="cri-o://5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c" gracePeriod=45
Sep 16 17:01:55.105985 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.105909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 17:01:55.239020 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.238977    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239020 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239021    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239264 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239060    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239264 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239129    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239264 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239188    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6rcs\" (UniqueName: \"kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239264 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm\") pod \"936ddc66-9bba-4a96-9470-889c5258e82b\" (UID: \"936ddc66-9bba-4a96-9470-889c5258e82b\") "
Sep 16 17:01:55.239505 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239344    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home" (OuterVolumeSpecName: "home") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:55.239505 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239357    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:55.239505 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache" (OuterVolumeSpecName: "model-cache") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:55.239681 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239548    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.239681 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239579    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.239681 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.239589    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.241415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.241373    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:01:55.241529 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.241468    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm" (OuterVolumeSpecName: "dshm") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:55.241529 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.241477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs" (OuterVolumeSpecName: "kube-api-access-z6rcs") pod "936ddc66-9bba-4a96-9470-889c5258e82b" (UID: "936ddc66-9bba-4a96-9470-889c5258e82b"). InnerVolumeSpecName "kube-api-access-z6rcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:01:55.340925 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.340884    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6rcs\" (UniqueName: \"kubernetes.io/projected/936ddc66-9bba-4a96-9470-889c5258e82b-kube-api-access-z6rcs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.340925 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.340921    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/936ddc66-9bba-4a96-9470-889c5258e82b-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.340925 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.340932    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/936ddc66-9bba-4a96-9470-889c5258e82b-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:55.365011 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.364922    2577 generic.go:356] "Generic (PLEG): container finished" podID="936ddc66-9bba-4a96-9470-889c5258e82b" containerID="5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c" exitCode=0
Sep 16 17:01:55.365011 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.364972    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"
Sep 16 17:01:55.365207 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.365005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" event={"ID":"936ddc66-9bba-4a96-9470-889c5258e82b","Type":"ContainerDied","Data":"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"}
Sep 16 17:01:55.365207 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.365053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr" event={"ID":"936ddc66-9bba-4a96-9470-889c5258e82b","Type":"ContainerDied","Data":"2c128b00ffb568dc7b35cc57f40080c32c8d616a1fad4a57d8dc324af6d66dfc"}
Sep 16 17:01:55.365207 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.365076    2577 scope.go:117] "RemoveContainer" containerID="5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"
Sep 16 17:01:55.374346 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.374327    2577 scope.go:117] "RemoveContainer" containerID="5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"
Sep 16 17:01:55.374681 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:01:55.374661    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c\": container with ID starting with 5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c not found: ID does not exist" containerID="5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"
Sep 16 17:01:55.374763 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.374692    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c"} err="failed to get container status \"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c\": rpc error: code = NotFound desc = could not find container \"5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c\": container with ID starting with 5a96d9ed458b80c90b5f3126b4dab89e97f73ca1fd8979376e78a7dd72177e5c not found: ID does not exist"
Sep 16 17:01:55.392276 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.392245    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 17:01:55.392276 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.392280    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/auth-test-model-52fd8c-kserve-5dd8bbfbf5-kk7dr"]
Sep 16 17:01:55.475599 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.475561    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:01:55.476195 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476178    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:01:55.476242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476199    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:01:55.476242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476221    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936ddc66-9bba-4a96-9470-889c5258e82b" containerName="main"
Sep 16 17:01:55.476242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476230    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="936ddc66-9bba-4a96-9470-889c5258e82b" containerName="main"
Sep 16 17:01:55.476341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476246    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" containerName="maas-api"
Sep 16 17:01:55.476341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476258    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" containerName="maas-api"
Sep 16 17:01:55.476341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8ac18d-681d-44c1-9dd3-d9be8f9cc408" containerName="cleanup"
Sep 16 17:01:55.476341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476290    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8ac18d-681d-44c1-9dd3-d9be8f9cc408" containerName="cleanup"
Sep 16 17:01:55.476490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476404    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="936ddc66-9bba-4a96-9470-889c5258e82b" containerName="main"
Sep 16 17:01:55.476490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476415    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8ac18d-681d-44c1-9dd3-d9be8f9cc408" containerName="cleanup"
Sep 16 17:01:55.476490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476426    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:01:55.476490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476436    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d62249-4aa0-40f3-bca2-3f89c12845ab" containerName="maas-api"
Sep 16 17:01:55.476490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.476447    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:01:55.481233 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.481211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.483800 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.483774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"default-dockercfg-hsq9j\""
Sep 16 17:01:55.484016 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.484001    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"kube-root-ca.crt\""
Sep 16 17:01:55.484097 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.484053    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"openshift-service-ca.crt\""
Sep 16 17:01:55.484398 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.484368    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-52fd8c\"/\"sub-test-model-52fd8c-kserve-self-signed-certs\""
Sep 16 17:01:55.494036 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.493987    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:01:55.542374 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.542575 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.542575 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.542575 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.542686 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.542686 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.542610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7m2\" (UniqueName: \"kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643439 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7m2\" (UniqueName: \"kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643439 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643692 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643692 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643692 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643692 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.643948 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.644014 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.644014 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.643996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.645743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.645727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.646054 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.646035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.652169 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.652136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7m2\" (UniqueName: \"kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2\") pod \"sub-test-model-52fd8c-kserve-5844449d4c-58r9s\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") " pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:55.786693 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.786650    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" podUID="67e5198f-d25f-4ee9-9106-696ad6173c0a" containerName="main" containerID="cri-o://27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366" gracePeriod=45
Sep 16 17:01:55.791169 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:55.791129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:01:56.045128 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.045098    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 17:01:56.068593 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.068561    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:01:56.071684 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:01:56.071652    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d3ef43b_ca8a_4d44_ba0f_79aecbbd7b5f.slice/crio-fd75d942b5b09bd90d289eb7ef42e7c2656ad52803d52671484e413e7621cda9 WatchSource:0}: Error finding container fd75d942b5b09bd90d289eb7ef42e7c2656ad52803d52671484e413e7621cda9: Status 404 returned error can't find the container with id fd75d942b5b09bd90d289eb7ef42e7c2656ad52803d52671484e413e7621cda9
Sep 16 17:01:56.149706 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149618    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.149706 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149695    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr8rl\" (UniqueName: \"kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.149929 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.149929 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.149929 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.149929 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.149870    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache\") pod \"67e5198f-d25f-4ee9-9106-696ad6173c0a\" (UID: \"67e5198f-d25f-4ee9-9106-696ad6173c0a\") "
Sep 16 17:01:56.150187 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.150157    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:56.150187 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.150181    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home" (OuterVolumeSpecName: "home") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:56.150312 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.150273    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache" (OuterVolumeSpecName: "model-cache") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:56.150400 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.150357    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.150400 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.150394    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.152099 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.152070    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm" (OuterVolumeSpecName: "dshm") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:01:56.152099 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.152102    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl" (OuterVolumeSpecName: "kube-api-access-fr8rl") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "kube-api-access-fr8rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:01:56.152271 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.152121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67e5198f-d25f-4ee9-9106-696ad6173c0a" (UID: "67e5198f-d25f-4ee9-9106-696ad6173c0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:01:56.251349 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.251307    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e5198f-d25f-4ee9-9106-696ad6173c0a-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.251349 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.251343    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr8rl\" (UniqueName: \"kubernetes.io/projected/67e5198f-d25f-4ee9-9106-696ad6173c0a-kube-api-access-fr8rl\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.251349 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.251352    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.251349 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.251360    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67e5198f-d25f-4ee9-9106-696ad6173c0a-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:01:56.372303 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.372263    2577 generic.go:356] "Generic (PLEG): container finished" podID="67e5198f-d25f-4ee9-9106-696ad6173c0a" containerID="27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366" exitCode=0
Sep 16 17:01:56.372542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.372312    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"
Sep 16 17:01:56.372542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.372347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" event={"ID":"67e5198f-d25f-4ee9-9106-696ad6173c0a","Type":"ContainerDied","Data":"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"}
Sep 16 17:01:56.372542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.372425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8" event={"ID":"67e5198f-d25f-4ee9-9106-696ad6173c0a","Type":"ContainerDied","Data":"46a832448d08eb7eec9b631bce392d3dc4d1f8eaf220ee9c32e2f446bfb9eb94"}
Sep 16 17:01:56.372542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.372452    2577 scope.go:117] "RemoveContainer" containerID="27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"
Sep 16 17:01:56.375289 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.375262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" event={"ID":"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f","Type":"ContainerStarted","Data":"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"}
Sep 16 17:01:56.375448 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.375305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" event={"ID":"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f","Type":"ContainerStarted","Data":"fd75d942b5b09bd90d289eb7ef42e7c2656ad52803d52671484e413e7621cda9"}
Sep 16 17:01:56.382058 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.382040    2577 scope.go:117] "RemoveContainer" containerID="27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"
Sep 16 17:01:56.382355 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:01:56.382336    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366\": container with ID starting with 27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366 not found: ID does not exist" containerID="27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"
Sep 16 17:01:56.382454 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.382364    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366"} err="failed to get container status \"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366\": rpc error: code = NotFound desc = could not find container \"27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366\": container with ID starting with 27eeac719daac5a8f87612065b92087f9bb761e588d5e738caabd490c9cec366 not found: ID does not exist"
Sep 16 17:01:56.393318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.393262    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" podStartSLOduration=1.39323386 podStartE2EDuration="1.39323386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:01:56.391055671 +0000 UTC m=+1533.476775744" watchObservedRunningTime="2026-09-16 17:01:56.39323386 +0000 UTC m=+1533.478953932"
Sep 16 17:01:56.406234 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.406151    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 17:01:56.410765 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:56.410733    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/auth-test-model-34d301-kserve-86b8df49bc-2xdt8"]
Sep 16 17:01:57.391150 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:57.391114    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e5198f-d25f-4ee9-9106-696ad6173c0a" path="/var/lib/kubelet/pods/67e5198f-d25f-4ee9-9106-696ad6173c0a/volumes"
Sep 16 17:01:57.391575 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:57.391540    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="936ddc66-9bba-4a96-9470-889c5258e82b" path="/var/lib/kubelet/pods/936ddc66-9bba-4a96-9470-889c5258e82b/volumes"
Sep 16 17:01:59.437946 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:01:59.437902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:02:05.792010 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:05.791967    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:02:05.792506 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:05.792025    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:02:05.804935 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:05.804899    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:02:06.450188 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:06.450153    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:02:20.591796 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.591749    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:02:20.592476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.592374    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:02:20.592476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.592413    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db8d2d4-9aac-4d71-8247-d92e4be3d555" containerName="istio-proxy"
Sep 16 17:02:20.592476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.592437    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e5198f-d25f-4ee9-9106-696ad6173c0a" containerName="main"
Sep 16 17:02:20.592476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.592446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e5198f-d25f-4ee9-9106-696ad6173c0a" containerName="main"
Sep 16 17:02:20.592713 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.592560    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e5198f-d25f-4ee9-9106-696ad6173c0a" containerName="main"
Sep 16 17:02:20.597731 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.597704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.600698 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.600669    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"default-dockercfg-vskgs\""
Sep 16 17:02:20.601062 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.600688    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"sub-test-model-34d301-kserve-self-signed-certs\""
Sep 16 17:02:20.601241 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.600699    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"openshift-service-ca.crt\""
Sep 16 17:02:20.601351 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.600751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"kube-root-ca.crt\""
Sep 16 17:02:20.602593 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.602572    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:02:20.706243 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.706243 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.706578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.706578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.706578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62vk\" (UniqueName: \"kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.706578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.706502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.807779 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.807991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62vk\" (UniqueName: \"kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808296 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.808231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808296 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.808283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.808379 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.808320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.810478 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.810447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.810718 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.810699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.816759 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.816728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62vk\" (UniqueName: \"kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk\") pod \"sub-test-model-34d301-kserve-7c7ccfff69-q99mp\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") " pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:20.909105 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:20.908989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:21.185352 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:21.185324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:02:21.187271 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:02:21.187239    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1df9d41d_c8f0_4a18_bf6a_d93621f344ba.slice/crio-feb6ccf3e92d8e9cbd1b05f3f221ce841761acf7e8ce7f2956b067da9f530d41 WatchSource:0}: Error finding container feb6ccf3e92d8e9cbd1b05f3f221ce841761acf7e8ce7f2956b067da9f530d41: Status 404 returned error can't find the container with id feb6ccf3e92d8e9cbd1b05f3f221ce841761acf7e8ce7f2956b067da9f530d41
Sep 16 17:02:21.529743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:21.529641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" event={"ID":"1df9d41d-c8f0-4a18-bf6a-d93621f344ba","Type":"ContainerStarted","Data":"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"}
Sep 16 17:02:21.529743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:21.529694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" event={"ID":"1df9d41d-c8f0-4a18-bf6a-d93621f344ba","Type":"ContainerStarted","Data":"feb6ccf3e92d8e9cbd1b05f3f221ce841761acf7e8ce7f2956b067da9f530d41"}
Sep 16 17:02:21.547216 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:21.547158    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" podStartSLOduration=1.547137641 podStartE2EDuration="1.547137641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:02:21.54530416 +0000 UTC m=+1558.631024233" watchObservedRunningTime="2026-09-16 17:02:21.547137641 +0000 UTC m=+1558.632857713"
Sep 16 17:02:23.822740 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.822499    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:02:23.831167 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.831134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.833723 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.833691    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9359a3-openshift-default-dockercfg-h674q\""
Sep 16 17:02:23.837745 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.837702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:02:23.941242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwmg\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:23.941592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:23.941472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042590 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwmg\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.042876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043259 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.042912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043259 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.043134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043377 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.043251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043377 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.043296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043514 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.043377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.043514 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.043477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.045328 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.045273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.046014 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.045991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.052044 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.052015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.052259 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.052236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwmg\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg\") pod \"e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") " pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.146316 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.146271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:24.494876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.494842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:02:24.496193 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:02:24.496165    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23072300_46b4_42c9_acde_cb5a5cbf57a1.slice/crio-c98fa530fab3233e7f5445e6923d694dc64416a3005bb6e29b47c3e23e0f125c WatchSource:0}: Error finding container c98fa530fab3233e7f5445e6923d694dc64416a3005bb6e29b47c3e23e0f125c: Status 404 returned error can't find the container with id c98fa530fab3233e7f5445e6923d694dc64416a3005bb6e29b47c3e23e0f125c
Sep 16 17:02:24.498938 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.498889    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:24.499031 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.498996    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:24.499075 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.499042    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:24.549293 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:24.549255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerStarted","Data":"c98fa530fab3233e7f5445e6923d694dc64416a3005bb6e29b47c3e23e0f125c"}
Sep 16 17:02:25.307397 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:25.307344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:02:25.556959 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:25.556927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerStarted","Data":"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"}
Sep 16 17:02:25.577948 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:25.577843    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podStartSLOduration=2.577823684 podStartE2EDuration="2.577823684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:02:25.57487487 +0000 UTC m=+1562.660594942" watchObservedRunningTime="2026-09-16 17:02:25.577823684 +0000 UTC m=+1562.663543757"
Sep 16 17:02:26.146587 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:26.146544    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:26.151817 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:26.151788    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:26.562918 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:26.562888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:26.564178 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:26.564155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:30.909199 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:30.909151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:30.909199 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:30.909213    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:30.923403 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:30.923350    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:31.602113 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:31.602082    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:02:37.301307 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:37.301266    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:02:47.691786 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.691698    2577 generic.go:356] "Generic (PLEG): container finished" podID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerID="a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002" exitCode=0
Sep 16 17:02:47.691786 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.691755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerDied","Data":"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"}
Sep 16 17:02:47.692277 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.692174    2577 scope.go:117] "RemoveContainer" containerID="a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"
Sep 16 17:02:47.693437 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.693399    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:47.693502 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.693472    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:47.693544 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:47.693502    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:02:48.699486 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:48.699452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerStarted","Data":"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"}
Sep 16 17:02:49.146985 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:49.146947    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:49.147207 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:49.147184    2577 patch_prober.go:28] interesting pod/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 16 17:02:49.147285 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:49.147242    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 16 17:02:50.147688 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:50.147649    2577 patch_prober.go:28] interesting pod/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 16 17:02:50.148112 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:50.147714    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 16 17:02:51.147013 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:51.146969    2577 patch_prober.go:28] interesting pod/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 16 17:02:51.147264 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:51.147040    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 16 17:02:52.147267 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:52.147221    2577 patch_prober.go:28] interesting pod/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 16 17:02:52.147809 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:52.147298    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 16 17:02:53.147041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:53.146997    2577 patch_prober.go:28] interesting pod/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 16 17:02:53.147468 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:53.147077    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 16 17:02:53.544253 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:53.544193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:02:54.151937 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:54.151884    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:54.152499 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:54.152411    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:02:54.153342 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:02:54.153319    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:03:09.695498 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.695459    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:03:09.698450 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.698423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.703629 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.703603    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4acea1-serving-cert\""
Sep 16 17:03:09.725547 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.725515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:03:09.842092 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.842047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.842297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.842121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-242l4\" (UniqueName: \"kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.842297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.842163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.842297 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.842187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.943463 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.943414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.943685 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.943480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-242l4\" (UniqueName: \"kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.943685 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.943520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.943685 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.943540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.943685 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:03:09.943585    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-serving-cert: secret "maas-api-e2e-api-b-4acea1-serving-cert" not found
Sep 16 17:03:09.943685 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:03:09.943671    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls podName:80113014-e4ce-414d-9aa4-439e238824ef nodeName:}" failed. No retries permitted until 2026-09-16 17:03:10.443648805 +0000 UTC m=+1607.529368863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls") pod "maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" (UID: "80113014-e4ce-414d-9aa4-439e238824ef") : secret "maas-api-e2e-api-b-4acea1-serving-cert" not found
Sep 16 17:03:09.944268 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.944246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.946262 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.946201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:09.953084 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:09.953052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-242l4\" (UniqueName: \"kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:10.448456 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:10.448376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:10.450956 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:10.450920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") pod \"maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:10.610531 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:10.610490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:10.902623 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:10.902592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:03:10.903799 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:03:10.903772    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80113014_e4ce_414d_9aa4_439e238824ef.slice/crio-fa0fe061129b1f3c2a92f413f8c8b6e71d03dc615c86485b1a071dcec8d69c77 WatchSource:0}: Error finding container fa0fe061129b1f3c2a92f413f8c8b6e71d03dc615c86485b1a071dcec8d69c77: Status 404 returned error can't find the container with id fa0fe061129b1f3c2a92f413f8c8b6e71d03dc615c86485b1a071dcec8d69c77
Sep 16 17:03:10.905101 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:10.905083    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:03:11.830841 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:11.830799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" event={"ID":"80113014-e4ce-414d-9aa4-439e238824ef","Type":"ContainerStarted","Data":"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"}
Sep 16 17:03:11.831054 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:11.830899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" event={"ID":"80113014-e4ce-414d-9aa4-439e238824ef","Type":"ContainerStarted","Data":"fa0fe061129b1f3c2a92f413f8c8b6e71d03dc615c86485b1a071dcec8d69c77"}
Sep 16 17:03:11.831054 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:11.830929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:11.853793 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:11.853731    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" podStartSLOduration=2.57250064 podStartE2EDuration="2.853712958s" totalImagesPullingTime="281.212318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:11.850852542 +0000 UTC m=+1608.936572835" watchObservedRunningTime="2026-09-16 17:03:11.853712958 +0000 UTC m=+1608.939433030"
Sep 16 17:03:12.121378 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:12.121273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:03:17.843027 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:17.842943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:03:21.587452 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:21.587411    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:03:22.684452 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:22.684407    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:03:28.218758 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:28.218718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:03:36.607443 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.607365    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" podUID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" containerName="main" containerID="cri-o://db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21" gracePeriod=45
Sep 16 17:03:36.873703 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.873619    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:03:36.937360 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937324    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.937606 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.937606 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937569    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg7m2\" (UniqueName: \"kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.937730 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.937730 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.937730 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937666    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:36.937730 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir\") pod \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\" (UID: \"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f\") "
Sep 16 17:03:36.938046 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:36.938046 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.937994    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home" (OuterVolumeSpecName: "home") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:36.938249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.938154    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:36.938249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.938174    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:36.938249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.938187    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:36.939786 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.939756    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:03:36.940186 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.940167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm" (OuterVolumeSpecName: "dshm") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:36.940272 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.940221    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2" (OuterVolumeSpecName: "kube-api-access-hg7m2") pod "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" (UID: "0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f"). InnerVolumeSpecName "kube-api-access-hg7m2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:36.978207 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.978171    2577 generic.go:356] "Generic (PLEG): container finished" podID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" containerID="db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21" exitCode=0
Sep 16 17:03:36.978419 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.978214    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"
Sep 16 17:03:36.978419 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.978255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" event={"ID":"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f","Type":"ContainerDied","Data":"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"}
Sep 16 17:03:36.978419 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.978299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s" event={"ID":"0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f","Type":"ContainerDied","Data":"fd75d942b5b09bd90d289eb7ef42e7c2656ad52803d52671484e413e7621cda9"}
Sep 16 17:03:36.978419 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.978322    2577 scope.go:117] "RemoveContainer" containerID="db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"
Sep 16 17:03:36.988450 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.988430    2577 scope.go:117] "RemoveContainer" containerID="db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"
Sep 16 17:03:36.988796 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:03:36.988771    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21\": container with ID starting with db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21 not found: ID does not exist" containerID="db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"
Sep 16 17:03:36.988881 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:36.988806    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21"} err="failed to get container status \"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21\": rpc error: code = NotFound desc = could not find container \"db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21\": container with ID starting with db4cb003a7282c02854913e29153bb7cddbc69c6db855bb3292240fe9854fa21 not found: ID does not exist"
Sep 16 17:03:37.003286 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.003252    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:03:37.010797 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.010760    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-52fd8c/sub-test-model-52fd8c-kserve-5844449d4c-58r9s"]
Sep 16 17:03:37.039221 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.039183    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:37.039221 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.039217    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg7m2\" (UniqueName: \"kubernetes.io/projected/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-kube-api-access-hg7m2\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:37.039221 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.039229    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:37.392365 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.392317    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" path="/var/lib/kubelet/pods/0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f/volumes"
Sep 16 17:03:37.703956 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.703851    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" podUID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" containerName="main" containerID="cri-o://67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0" gracePeriod=45
Sep 16 17:03:37.976795 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.976763    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:03:37.986367 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.986336    2577 generic.go:356] "Generic (PLEG): container finished" podID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" containerID="67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0" exitCode=0
Sep 16 17:03:37.986564 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.986376    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"
Sep 16 17:03:37.986564 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.986422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" event={"ID":"1df9d41d-c8f0-4a18-bf6a-d93621f344ba","Type":"ContainerDied","Data":"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"}
Sep 16 17:03:37.986564 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.986471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp" event={"ID":"1df9d41d-c8f0-4a18-bf6a-d93621f344ba","Type":"ContainerDied","Data":"feb6ccf3e92d8e9cbd1b05f3f221ce841761acf7e8ce7f2956b067da9f530d41"}
Sep 16 17:03:37.986564 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.986489    2577 scope.go:117] "RemoveContainer" containerID="67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"
Sep 16 17:03:37.995478 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.995454    2577 scope.go:117] "RemoveContainer" containerID="67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"
Sep 16 17:03:37.995830 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:03:37.995805    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0\": container with ID starting with 67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0 not found: ID does not exist" containerID="67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"
Sep 16 17:03:37.995886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:37.995840    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0"} err="failed to get container status \"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0\": rpc error: code = NotFound desc = could not find container \"67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0\": container with ID starting with 67bb220dbf78525d83f8b7f3a080284b8a7e8cba45418c5125b77414078a44b0 not found: ID does not exist"
Sep 16 17:03:38.049814 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.049782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050015 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.049835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050015 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.049884    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050015 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.049970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050015 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050248 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n62vk\" (UniqueName: \"kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk\") pod \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\" (UID: \"1df9d41d-c8f0-4a18-bf6a-d93621f344ba\") "
Sep 16 17:03:38.050248 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050186    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:38.050248 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:38.050436 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050415    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.050487 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050427    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home" (OuterVolumeSpecName: "home") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:38.050487 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.050444    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.052266 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.052233    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk" (OuterVolumeSpecName: "kube-api-access-n62vk") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "kube-api-access-n62vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:38.052494 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.052472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm" (OuterVolumeSpecName: "dshm") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:03:38.052565 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.052528    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1df9d41d-c8f0-4a18-bf6a-d93621f344ba" (UID: "1df9d41d-c8f0-4a18-bf6a-d93621f344ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:03:38.151318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.151272    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.151318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.151308    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.151318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.151319    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n62vk\" (UniqueName: \"kubernetes.io/projected/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-kube-api-access-n62vk\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.151318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.151330    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df9d41d-c8f0-4a18-bf6a-d93621f344ba-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:03:38.311943 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.311903    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:03:38.315933 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:38.315905    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/sub-test-model-34d301-kserve-7c7ccfff69-q99mp"]
Sep 16 17:03:39.392345 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:39.392306    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" path="/var/lib/kubelet/pods/1df9d41d-c8f0-4a18-bf6a-d93621f344ba/volumes"
Sep 16 17:03:45.618547 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:03:45.618508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:01.003919 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:01.003878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:02.349116 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:02.349072    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:04:02.349643 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:02.349375    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy" containerID="cri-o://d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683" gracePeriod=30
Sep 16 17:04:04.979762 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.979715    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:04:04.980208 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980192    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" containerName="main"
Sep 16 17:04:04.980260 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980211    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" containerName="main"
Sep 16 17:04:04.980260 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980242    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" containerName="main"
Sep 16 17:04:04.980260 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980248    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" containerName="main"
Sep 16 17:04:04.980807 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980316    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1df9d41d-c8f0-4a18-bf6a-d93621f344ba" containerName="main"
Sep 16 17:04:04.980807 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.980326    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3ef43b-ca8a-4d44-ba0f-79aecbbd7b5f" containerName="main"
Sep 16 17:04:04.983703 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.983668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:04.986097 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.986068    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"openshift-service-ca.crt\""
Sep 16 17:04:04.986230 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.986133    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"rate-test-model-34d301-kserve-self-signed-certs\""
Sep 16 17:04:04.987142 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.987120    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"default-dockercfg-vskgs\""
Sep 16 17:04:04.987232 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.987147    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34d301\"/\"kube-root-ca.crt\""
Sep 16 17:04:04.994434 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:04.994351    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:04:05.031261 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.031488 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfnl\" (UniqueName: \"kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.031488 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.031488 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.031488 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.031754 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.031496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132662 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfnl\" (UniqueName: \"kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.132886 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.132820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.133176 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.133141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.133176 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.133163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.133261 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.133241    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.135223 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.135204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.135427 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.135410    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.146757 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.146726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfnl\" (UniqueName: \"kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl\") pod \"rate-test-model-34d301-kserve-7b44854d7f-mdcpx\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") " pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.299458 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.299421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:05.598280 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:05.598247    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:04:05.600557 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:04:05.600527    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4ad3e11_61b8_4ce7_90f9_955795484b8b.slice/crio-e5cf9d2f2fce87b1a32bb9571e332e3a4b924b87670377584794eb242daa2cc1 WatchSource:0}: Error finding container e5cf9d2f2fce87b1a32bb9571e332e3a4b924b87670377584794eb242daa2cc1: Status 404 returned error can't find the container with id e5cf9d2f2fce87b1a32bb9571e332e3a4b924b87670377584794eb242daa2cc1
Sep 16 17:04:06.148987 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:06.148951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" event={"ID":"a4ad3e11-61b8-4ce7-90f9-955795484b8b","Type":"ContainerStarted","Data":"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"}
Sep 16 17:04:06.149441 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:06.149000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" event={"ID":"a4ad3e11-61b8-4ce7-90f9-955795484b8b","Type":"ContainerStarted","Data":"e5cf9d2f2fce87b1a32bb9571e332e3a4b924b87670377584794eb242daa2cc1"}
Sep 16 17:04:06.167566 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:06.167509    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" podStartSLOduration=2.167490394 podStartE2EDuration="2.167490394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:04:06.164083064 +0000 UTC m=+1663.249803135" watchObservedRunningTime="2026-09-16 17:04:06.167490394 +0000 UTC m=+1663.253210472"
Sep 16 17:04:07.618767 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.618686    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:04:07.658443 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658406    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zwmg\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.658994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658762    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:04:07.658994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:04:07.658994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket\") pod \"23072300-46b4-42c9-acde-cb5a5cbf57a1\" (UID: \"23072300-46b4-42c9-acde-cb5a5cbf57a1\") "
Sep 16 17:04:07.659208 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.658996    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:04:07.659208 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659142    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:07.659315 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659301    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:04:07.659472 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659455    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.659546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659477    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.659546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659491    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23072300-46b4-42c9-acde-cb5a5cbf57a1-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.659546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659504    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.659546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.659516    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.661107 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.661082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:04:07.661376 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.661346    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:04:07.661376 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.661401    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg" (OuterVolumeSpecName: "kube-api-access-9zwmg") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "kube-api-access-9zwmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:07.661591 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.661431    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "23072300-46b4-42c9-acde-cb5a5cbf57a1" (UID: "23072300-46b4-42c9-acde-cb5a5cbf57a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:07.760090 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.760046    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zwmg\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-kube-api-access-9zwmg\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.760090 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.760085    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.760090 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.760095    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:07.760090 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:07.760102    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23072300-46b4-42c9-acde-cb5a5cbf57a1-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:08.161756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.161723    2577 generic.go:356] "Generic (PLEG): container finished" podID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerID="d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683" exitCode=0
Sep 16 17:04:08.161980 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.161780    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"
Sep 16 17:04:08.161980 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.161810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerDied","Data":"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"}
Sep 16 17:04:08.161980 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.161872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8" event={"ID":"23072300-46b4-42c9-acde-cb5a5cbf57a1","Type":"ContainerDied","Data":"c98fa530fab3233e7f5445e6923d694dc64416a3005bb6e29b47c3e23e0f125c"}
Sep 16 17:04:08.161980 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.161895    2577 scope.go:117] "RemoveContainer" containerID="d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"
Sep 16 17:04:08.172107 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.172081    2577 scope.go:117] "RemoveContainer" containerID="a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"
Sep 16 17:04:08.181294 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.181273    2577 scope.go:117] "RemoveContainer" containerID="d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"
Sep 16 17:04:08.181691 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:04:08.181659    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683\": container with ID starting with d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683 not found: ID does not exist" containerID="d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"
Sep 16 17:04:08.181889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.181703    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683"} err="failed to get container status \"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683\": rpc error: code = NotFound desc = could not find container \"d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683\": container with ID starting with d0f093a13770b99f868018662394e98c6a1eba8a3d19f458199f8215d3db2683 not found: ID does not exist"
Sep 16 17:04:08.181889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.181731    2577 scope.go:117] "RemoveContainer" containerID="a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"
Sep 16 17:04:08.182549 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:04:08.182514    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002\": container with ID starting with a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002 not found: ID does not exist" containerID="a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"
Sep 16 17:04:08.182676 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.182553    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002"} err="failed to get container status \"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002\": rpc error: code = NotFound desc = could not find container \"a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002\": container with ID starting with a2c66b78f77599d263c3f6f86680e8fb55c745692a5c8f9fa292b8d312879002 not found: ID does not exist"
Sep 16 17:04:08.185009 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.184987    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:04:08.190848 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:08.190823    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9359a3-openshift-default-789fd784b4-2frh8"]
Sep 16 17:04:09.391768 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:09.391731    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" path="/var/lib/kubelet/pods/23072300-46b4-42c9-acde-cb5a5cbf57a1/volumes"
Sep 16 17:04:12.604863 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:12.604816    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:15.299826 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:15.299787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:15.300267 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:15.299952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:15.313050 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:15.313014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:16.227512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:16.227480    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:04:18.298418 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.298360    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:04:18.299052 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.298598    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" podUID="80113014-e4ce-414d-9aa4-439e238824ef" containerName="maas-api" containerID="cri-o://13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74" gracePeriod=30
Sep 16 17:04:18.564555 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.564488    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:04:18.672699 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.672663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") pod \"80113014-e4ce-414d-9aa4-439e238824ef\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") "
Sep 16 17:04:18.672889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.672714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs\") pod \"80113014-e4ce-414d-9aa4-439e238824ef\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") "
Sep 16 17:04:18.672889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.672760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-242l4\" (UniqueName: \"kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4\") pod \"80113014-e4ce-414d-9aa4-439e238824ef\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") "
Sep 16 17:04:18.673050 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.672950    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca\") pod \"80113014-e4ce-414d-9aa4-439e238824ef\" (UID: \"80113014-e4ce-414d-9aa4-439e238824ef\") "
Sep 16 17:04:18.673327 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.673303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80113014-e4ce-414d-9aa4-439e238824ef" (UID: "80113014-e4ce-414d-9aa4-439e238824ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:18.673476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.673425    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80113014-e4ce-414d-9aa4-439e238824ef-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.675136 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.675103    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4" (OuterVolumeSpecName: "kube-api-access-242l4") pod "80113014-e4ce-414d-9aa4-439e238824ef" (UID: "80113014-e4ce-414d-9aa4-439e238824ef"). InnerVolumeSpecName "kube-api-access-242l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:18.675855 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.675827    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80113014-e4ce-414d-9aa4-439e238824ef" (UID: "80113014-e4ce-414d-9aa4-439e238824ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:04:18.675855 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.675847    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "80113014-e4ce-414d-9aa4-439e238824ef" (UID: "80113014-e4ce-414d-9aa4-439e238824ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:04:18.774330 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.774285    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.774330 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.774322    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80113014-e4ce-414d-9aa4-439e238824ef-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.774330 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:18.774333    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-242l4\" (UniqueName: \"kubernetes.io/projected/80113014-e4ce-414d-9aa4-439e238824ef-kube-api-access-242l4\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:04:19.233282 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.233247    2577 generic.go:356] "Generic (PLEG): container finished" podID="80113014-e4ce-414d-9aa4-439e238824ef" containerID="13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74" exitCode=0
Sep 16 17:04:19.233498 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.233297    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"
Sep 16 17:04:19.233498 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.233317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" event={"ID":"80113014-e4ce-414d-9aa4-439e238824ef","Type":"ContainerDied","Data":"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"}
Sep 16 17:04:19.233498 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.233424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr" event={"ID":"80113014-e4ce-414d-9aa4-439e238824ef","Type":"ContainerDied","Data":"fa0fe061129b1f3c2a92f413f8c8b6e71d03dc615c86485b1a071dcec8d69c77"}
Sep 16 17:04:19.233498 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.233448    2577 scope.go:117] "RemoveContainer" containerID="13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"
Sep 16 17:04:19.243661 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.243635    2577 scope.go:117] "RemoveContainer" containerID="13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"
Sep 16 17:04:19.244004 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:04:19.243982    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74\": container with ID starting with 13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74 not found: ID does not exist" containerID="13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"
Sep 16 17:04:19.244059 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.244017    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74"} err="failed to get container status \"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74\": rpc error: code = NotFound desc = could not find container \"13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74\": container with ID starting with 13d8d1118129514886e7eb1f49648dcf8bb70975d872e642df90f70d032edb74 not found: ID does not exist"
Sep 16 17:04:19.256124 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.256068    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:04:19.258512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.258480    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4acea1-fc9c8cb8f-c7hsr"]
Sep 16 17:04:19.391573 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:19.391539    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80113014-e4ce-414d-9aa4-439e238824ef" path="/var/lib/kubelet/pods/80113014-e4ce-414d-9aa4-439e238824ef/volumes"
Sep 16 17:04:26.015405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:26.015341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:37.915202 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:37.915165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:52.318913 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:52.318869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:04:58.181425 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:04:58.181374    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:05:01.187516 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:01.187473    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 17:05:01.188059 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:01.187723    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy" containerID="cri-o://ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb" gracePeriod=30
Sep 16 17:05:06.465464 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.465373    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 17:05:06.474215 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474189    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474418 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474350    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474418 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474406    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474508 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474548 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474506    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:06.474548 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474561    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:06.474657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474643    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5slvb\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb\") pod \"24ad5601-3f8b-4e88-8aec-c827863223cd\" (UID: \"24ad5601-3f8b-4e88-8aec-c827863223cd\") "
Sep 16 17:05:06.474870 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:06.474870 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.474849    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:05:06.475318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.475054    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ad5601-3f8b-4e88-8aec-c827863223cd-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.475318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.475077    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.475318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.475091    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.475318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.475098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:06.475318 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.475105    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.476783 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.476754    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:05:06.476898 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.476779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:05:06.476898 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.476825    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:06.477142 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.477125    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb" (OuterVolumeSpecName: "kube-api-access-5slvb") pod "24ad5601-3f8b-4e88-8aec-c827863223cd" (UID: "24ad5601-3f8b-4e88-8aec-c827863223cd"). InnerVolumeSpecName "kube-api-access-5slvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:05:06.498741 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.498703    2577 generic.go:356] "Generic (PLEG): container finished" podID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerID="ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb" exitCode=0
Sep 16 17:05:06.498922 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.498752    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"
Sep 16 17:05:06.498922 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.498785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerDied","Data":"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"}
Sep 16 17:05:06.498922 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.498845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf" event={"ID":"24ad5601-3f8b-4e88-8aec-c827863223cd","Type":"ContainerDied","Data":"a0c6e05d1eff63a7d3acbf717b0e3b0bdb71d7ab75fb6bd665f3aec5af669ec9"}
Sep 16 17:05:06.498922 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.498874    2577 scope.go:117] "RemoveContainer" containerID="ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"
Sep 16 17:05:06.516512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.516481    2577 scope.go:117] "RemoveContainer" containerID="29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"
Sep 16 17:05:06.525686 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.525659    2577 scope.go:117] "RemoveContainer" containerID="ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"
Sep 16 17:05:06.526032 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:05:06.526009    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb\": container with ID starting with ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb not found: ID does not exist" containerID="ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"
Sep 16 17:05:06.526092 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.526043    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb"} err="failed to get container status \"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb\": rpc error: code = NotFound desc = could not find container \"ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb\": container with ID starting with ead95dc3180bf42d304e4e515d2e6c9cb4986e5cdd84b952f762b19d9459ffdb not found: ID does not exist"
Sep 16 17:05:06.526092 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.526060    2577 scope.go:117] "RemoveContainer" containerID="29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"
Sep 16 17:05:06.526363 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:05:06.526343    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba\": container with ID starting with 29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba not found: ID does not exist" containerID="29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"
Sep 16 17:05:06.526497 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.526371    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba"} err="failed to get container status \"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba\": rpc error: code = NotFound desc = could not find container \"29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba\": container with ID starting with 29baefcc83b558dd8bad7dfd3684b0b829d93d33d9efdd9f30b2715624294cba not found: ID does not exist"
Sep 16 17:05:06.531235 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.531206    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 17:05:06.535178 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.535149    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2ce8bc-openshift-default-86ccb8b49d-fhqnf"]
Sep 16 17:05:06.576322 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.576285    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.576322 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.576314    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ad5601-3f8b-4e88-8aec-c827863223cd-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.576322 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.576324    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5slvb\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-kube-api-access-5slvb\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.576581 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.576354    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:06.576581 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:06.576364    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ad5601-3f8b-4e88-8aec-c827863223cd-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:07.391121 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:07.391084    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" path="/var/lib/kubelet/pods/24ad5601-3f8b-4e88-8aec-c827863223cd/volumes"
Sep 16 17:05:07.810463 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:07.810424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:05:13.201507 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.201438    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" podUID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" containerName="main" containerID="cri-o://185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667" gracePeriod=45
Sep 16 17:05:13.459269 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.459194    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:05:13.538878 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.538840    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" containerID="185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667" exitCode=0
Sep 16 17:05:13.539053 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.538885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"
Sep 16 17:05:13.539053 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.538886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" event={"ID":"a4ad3e11-61b8-4ce7-90f9-955795484b8b","Type":"ContainerDied","Data":"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"}
Sep 16 17:05:13.539053 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.539008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx" event={"ID":"a4ad3e11-61b8-4ce7-90f9-955795484b8b","Type":"ContainerDied","Data":"e5cf9d2f2fce87b1a32bb9571e332e3a4b924b87670377584794eb242daa2cc1"}
Sep 16 17:05:13.539053 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.539035    2577 scope.go:117] "RemoveContainer" containerID="185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"
Sep 16 17:05:13.547771 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.547751    2577 scope.go:117] "RemoveContainer" containerID="185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"
Sep 16 17:05:13.548137 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:05:13.548113    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667\": container with ID starting with 185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667 not found: ID does not exist" containerID="185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"
Sep 16 17:05:13.548218 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.548147    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667"} err="failed to get container status \"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667\": rpc error: code = NotFound desc = could not find container \"185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667\": container with ID starting with 185d64c7c137c8f2cd5cf1ac3894be60863a6bd16af8c8c35f65af73d2374667 not found: ID does not exist"
Sep 16 17:05:13.553524 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553612 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553560    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553612 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553612 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553607    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553722 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553722 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdfnl\" (UniqueName: \"kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl\") pod \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\" (UID: \"a4ad3e11-61b8-4ce7-90f9-955795484b8b\") "
Sep 16 17:05:13.553843 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:13.553897 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553854    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:13.553954 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.553899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home" (OuterVolumeSpecName: "home") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:13.554127 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.554105    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.554127 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.554125    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.554309 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.554134    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.555793 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.555769    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm" (OuterVolumeSpecName: "dshm") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:05:13.555889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.555837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:05:13.555889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.555834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl" (OuterVolumeSpecName: "kube-api-access-gdfnl") pod "a4ad3e11-61b8-4ce7-90f9-955795484b8b" (UID: "a4ad3e11-61b8-4ce7-90f9-955795484b8b"). InnerVolumeSpecName "kube-api-access-gdfnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:05:13.655268 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.655223    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ad3e11-61b8-4ce7-90f9-955795484b8b-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.655268 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.655256    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ad3e11-61b8-4ce7-90f9-955795484b8b-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.655268 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.655268    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdfnl\" (UniqueName: \"kubernetes.io/projected/a4ad3e11-61b8-4ce7-90f9-955795484b8b-kube-api-access-gdfnl\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:05:13.862742 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.862712    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:05:13.869358 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:13.869326    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34d301/rate-test-model-34d301-kserve-7b44854d7f-mdcpx"]
Sep 16 17:05:15.391604 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:15.391567    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" path="/var/lib/kubelet/pods/a4ad3e11-61b8-4ce7-90f9-955795484b8b/volumes"
Sep 16 17:05:28.300722 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.300680    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301087    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301099    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301118    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301123    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301129    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" containerName="main"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301136    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" containerName="main"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301155    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80113014-e4ce-414d-9aa4-439e238824ef" containerName="maas-api"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301160    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="80113014-e4ce-414d-9aa4-439e238824ef" containerName="maas-api"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301222    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301231    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="80113014-e4ce-414d-9aa4-439e238824ef" containerName="maas-api"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301239    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ad3e11-61b8-4ce7-90f9-955795484b8b" containerName="main"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301245    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:28.301331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.301251    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:28.304331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.304305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.308042 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.308016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6586c3-openshift-default-dockercfg-pwlvg\""
Sep 16 17:05:28.321066 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.321027    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:05:28.399958 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.399906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400188 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.399967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400188 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400188 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kmd\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400188 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.400476 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.400373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501723 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501723 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501974 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501974 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kmd\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501974 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501974 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.501974 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.501972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502095    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502441 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502499 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502560 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502498    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.502860 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.502838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.504096 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.504071    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.504320 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.504303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.513713 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.513684    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.513889 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.513867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kmd\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd\") pod \"e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") " pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.616544 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.616456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:28.899015 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.898925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:05:28.903275 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:05:28.903248    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd757286_0a59_465b_b7c5_8670a5f60153.slice/crio-3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368 WatchSource:0}: Error finding container 3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368: Status 404 returned error can't find the container with id 3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368
Sep 16 17:05:28.905630 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.905585    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:28.905743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.905680    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:28.905743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:28.905720    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:29.628637 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:29.628601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerStarted","Data":"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"}
Sep 16 17:05:29.629059 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:29.628653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerStarted","Data":"3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368"}
Sep 16 17:05:29.657490 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:29.657431    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podStartSLOduration=1.657410944 podStartE2EDuration="1.657410944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:05:29.655258626 +0000 UTC m=+1746.740978697" watchObservedRunningTime="2026-09-16 17:05:29.657410944 +0000 UTC m=+1746.743131013"
Sep 16 17:05:30.617550 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:30.617506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:30.623140 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:30.623107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:30.634520 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:30.634495    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:30.635597 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:30.635575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:34.449551 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.449511    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:05:34.450166 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.450145    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:34.450251 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.450169    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:34.450251 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.450194    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:34.450251 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.450202    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ad5601-3f8b-4e88-8aec-c827863223cd" containerName="istio-proxy"
Sep 16 17:05:34.450442 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.450290    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="23072300-46b4-42c9-acde-cb5a5cbf57a1" containerName="istio-proxy"
Sep 16 17:05:34.453648 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.453623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.456046 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.456007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6586c3-serving-cert\""
Sep 16 17:05:34.465347 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.465120    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:05:34.564158 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.564096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.564158 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.564156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4j56\" (UniqueName: \"kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.564432 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.564263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.564432 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.564309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.665223 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.665186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.665223 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.665236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.665499 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.665309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.665499 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.665338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4j56\" (UniqueName: \"kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.665499 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:05:34.665360    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-serving-cert: secret "maas-api-e2e-ipp-b-6586c3-serving-cert" not found
Sep 16 17:05:34.665499 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:05:34.665448    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls podName:0ef3ac09-241b-4600-b71c-72ca74858ef2 nodeName:}" failed. No retries permitted until 2026-09-16 17:05:35.165427305 +0000 UTC m=+1752.251147362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls") pod "maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" (UID: "0ef3ac09-241b-4600-b71c-72ca74858ef2") : secret "maas-api-e2e-ipp-b-6586c3-serving-cert" not found
Sep 16 17:05:34.666061 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.666035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.667869 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.667842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:34.673731 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:34.673701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4j56\" (UniqueName: \"kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:35.170781 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:35.170724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:35.173281 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:35.173251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:35.369056 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:35.369019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:35.647097 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:35.646995    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:05:35.650856 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:05:35.650816    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef3ac09_241b_4600_b71c_72ca74858ef2.slice/crio-2ae81766258b296bf6a85b75bee704fe8f92576fd6157c9c645f2660271d3380 WatchSource:0}: Error finding container 2ae81766258b296bf6a85b75bee704fe8f92576fd6157c9c645f2660271d3380: Status 404 returned error can't find the container with id 2ae81766258b296bf6a85b75bee704fe8f92576fd6157c9c645f2660271d3380
Sep 16 17:05:35.662625 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:35.662592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" event={"ID":"0ef3ac09-241b-4600-b71c-72ca74858ef2","Type":"ContainerStarted","Data":"2ae81766258b296bf6a85b75bee704fe8f92576fd6157c9c645f2660271d3380"}
Sep 16 17:05:36.669912 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:36.669875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" event={"ID":"0ef3ac09-241b-4600-b71c-72ca74858ef2","Type":"ContainerStarted","Data":"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"}
Sep 16 17:05:36.670362 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:36.670013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:39.314825 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:39.314767    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" podStartSLOduration=4.887032413 podStartE2EDuration="5.314745837s" totalImagesPullingTime="427.713424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:05:36.686209551 +0000 UTC m=+1753.771929620" watchObservedRunningTime="2026-09-16 17:05:39.314745837 +0000 UTC m=+1756.400465913"
Sep 16 17:05:39.316769 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:39.316748    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:05:42.680650 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:42.680618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:05:50.755499 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.755458    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd757286-0a59-465b-b7c5-8670a5f60153" containerID="a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f" exitCode=0
Sep 16 17:05:50.755948 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.755532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerDied","Data":"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"}
Sep 16 17:05:50.756002 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.755987    2577 scope.go:117] "RemoveContainer" containerID="a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"
Sep 16 17:05:50.757178 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.757148    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:50.757242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.757217    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:50.757287 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:50.757252    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 17:05:51.763607 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:51.763570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerStarted","Data":"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"}
Sep 16 17:05:52.616662 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:52.616625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:52.616906 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:52.616841    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 16 17:05:52.616906 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:52.616892    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 16 17:05:53.617864 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:53.617825    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 16 17:05:53.618331 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:53.617888    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 16 17:05:54.617901 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:54.617860    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 16 17:05:54.618455 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:54.617927    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 16 17:05:55.617366 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:55.617330    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 16 17:05:55.617578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:55.617424    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 16 17:05:56.616930 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:56.616889    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 16 17:05:56.617478 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:56.616959    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 16 17:05:57.621648 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:57.621618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:57.622127 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:57.621934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:05:57.622799 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:05:57.622781    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:06:26.282689 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.282621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:06:26.286373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.286348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.289519 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.289491    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-9ac315\"/\"default-dockercfg-sht6d\""
Sep 16 17:06:26.289693 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.289564    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-9ac315\"/\"kube-root-ca.crt\""
Sep 16 17:06:26.289693 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.289622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-9ac315\"/\"openshift-service-ca.crt\""
Sep 16 17:06:26.289693 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.289639    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-9ac315\"/\"ipp-route-9ac315-kserve-self-signed-certs\""
Sep 16 17:06:26.295521 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.295490    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:06:26.415569 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.415756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.415756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.415756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.415756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nr4r\" (UniqueName: \"kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.415900 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.415772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.516972 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.516924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.516972 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.516972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.516972 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.516997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nr4r\" (UniqueName: \"kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517430 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517611 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517663 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.517769 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.517748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.519567 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.519544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.519884 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.519859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.525193 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.525168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nr4r\" (UniqueName: \"kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r\") pod \"ipp-route-9ac315-kserve-5f57976c47-jnjkb\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") " pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.597113 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.597070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:26.873416 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.873357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:06:26.875337 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:06:26.875312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63f26443_0f6b_4d93_9eb0_fc998bbb0077.slice/crio-114bf44ad28a7107bafd7a51ee9087745a221a01c8e5d1e435c2be642b9c9eac WatchSource:0}: Error finding container 114bf44ad28a7107bafd7a51ee9087745a221a01c8e5d1e435c2be642b9c9eac: Status 404 returned error can't find the container with id 114bf44ad28a7107bafd7a51ee9087745a221a01c8e5d1e435c2be642b9c9eac
Sep 16 17:06:26.949060 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:26.949025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" event={"ID":"63f26443-0f6b-4d93-9eb0-fc998bbb0077","Type":"ContainerStarted","Data":"114bf44ad28a7107bafd7a51ee9087745a221a01c8e5d1e435c2be642b9c9eac"}
Sep 16 17:06:27.956200 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:27.956165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" event={"ID":"63f26443-0f6b-4d93-9eb0-fc998bbb0077","Type":"ContainerStarted","Data":"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"}
Sep 16 17:06:27.972338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:27.972283    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" podStartSLOduration=1.972265906 podStartE2EDuration="1.972265906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:06:27.971352348 +0000 UTC m=+1805.057072422" watchObservedRunningTime="2026-09-16 17:06:27.972265906 +0000 UTC m=+1805.057985977"
Sep 16 17:06:30.204892 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:30.204858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:06:36.597448 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:36.597363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:36.597902 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:36.597463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:36.610208 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:36.610178    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:37.015693 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:37.015596    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:06:46.210801 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:46.210759    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:06:59.571292 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.571255    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:06:59.575442 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.575414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.577742 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.577711    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-48fbf0-serving-cert\""
Sep 16 17:06:59.586337 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.586262    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:06:59.667657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.667600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.667846 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.667686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.667846 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.667723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hx6s\" (UniqueName: \"kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.667846 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.667740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.768731 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.768691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.768731 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.768733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hx6s\" (UniqueName: \"kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.769029 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.768760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.769029 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.768874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.769553 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.769531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.771330 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.771304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.771422 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.771358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.777950 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.777924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hx6s\" (UniqueName: \"kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s\") pod \"maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:06:59.890557 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:06:59.890466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:07:00.193413 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:00.193344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:07:00.196734 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:07:00.196707    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c986fb_859c_4429_9b8f_2c69cf64ce6c.slice/crio-c147531a9038510a352a5d434b8de732555968ab59e4518d3c0a5fe36270bcfc WatchSource:0}: Error finding container c147531a9038510a352a5d434b8de732555968ab59e4518d3c0a5fe36270bcfc: Status 404 returned error can't find the container with id c147531a9038510a352a5d434b8de732555968ab59e4518d3c0a5fe36270bcfc
Sep 16 17:07:01.152121 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:01.152068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" event={"ID":"a4c986fb-859c-4429-9b8f-2c69cf64ce6c","Type":"ContainerStarted","Data":"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"}
Sep 16 17:07:01.152596 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:01.152140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:07:01.152596 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:01.152158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" event={"ID":"a4c986fb-859c-4429-9b8f-2c69cf64ce6c","Type":"ContainerStarted","Data":"c147531a9038510a352a5d434b8de732555968ab59e4518d3c0a5fe36270bcfc"}
Sep 16 17:07:01.170621 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:01.170561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" podStartSLOduration=1.743607701 podStartE2EDuration="2.170543607s" totalImagesPullingTime="426.935906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:07:01.168877195 +0000 UTC m=+1838.254597268" watchObservedRunningTime="2026-09-16 17:07:01.170543607 +0000 UTC m=+1838.256263678"
Sep 16 17:07:05.513880 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:05.513841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:07:07.163723 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:07.163692    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:07:37.823755 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:37.823702    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:07:37.824648 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:37.823954    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" podUID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" containerName="maas-api" containerID="cri-o://4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f" gracePeriod=30
Sep 16 17:07:38.082523 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.082447    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:07:38.175729 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.175692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs\") pod \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") "
Sep 16 17:07:38.176004 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.175741    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls\") pod \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") "
Sep 16 17:07:38.176004 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.175780    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hx6s\" (UniqueName: \"kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s\") pod \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") "
Sep 16 17:07:38.176004 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.175839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca\") pod \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\" (UID: \"a4c986fb-859c-4429-9b8f-2c69cf64ce6c\") "
Sep 16 17:07:38.176241 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.176202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4c986fb-859c-4429-9b8f-2c69cf64ce6c" (UID: "a4c986fb-859c-4429-9b8f-2c69cf64ce6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:38.177993 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.177964    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a4c986fb-859c-4429-9b8f-2c69cf64ce6c" (UID: "a4c986fb-859c-4429-9b8f-2c69cf64ce6c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:07:38.178129 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.177995    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s" (OuterVolumeSpecName: "kube-api-access-8hx6s") pod "a4c986fb-859c-4429-9b8f-2c69cf64ce6c" (UID: "a4c986fb-859c-4429-9b8f-2c69cf64ce6c"). InnerVolumeSpecName "kube-api-access-8hx6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:38.178129 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.178080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4c986fb-859c-4429-9b8f-2c69cf64ce6c" (UID: "a4c986fb-859c-4429-9b8f-2c69cf64ce6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:07:38.277291 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.277227    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.277291 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.277277    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.277291 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.277287    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hx6s\" (UniqueName: \"kubernetes.io/projected/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-kube-api-access-8hx6s\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.277291 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.277297    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c986fb-859c-4429-9b8f-2c69cf64ce6c-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.359453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.359337    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" containerID="4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f" exitCode=0
Sep 16 17:07:38.359453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.359404    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"
Sep 16 17:07:38.359453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.359428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" event={"ID":"a4c986fb-859c-4429-9b8f-2c69cf64ce6c","Type":"ContainerDied","Data":"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"}
Sep 16 17:07:38.359726 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.359491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k" event={"ID":"a4c986fb-859c-4429-9b8f-2c69cf64ce6c","Type":"ContainerDied","Data":"c147531a9038510a352a5d434b8de732555968ab59e4518d3c0a5fe36270bcfc"}
Sep 16 17:07:38.359726 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.359524    2577 scope.go:117] "RemoveContainer" containerID="4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"
Sep 16 17:07:38.372722 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.372697    2577 scope.go:117] "RemoveContainer" containerID="4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"
Sep 16 17:07:38.373122 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:07:38.373094    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f\": container with ID starting with 4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f not found: ID does not exist" containerID="4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"
Sep 16 17:07:38.373224 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.373140    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f"} err="failed to get container status \"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f\": rpc error: code = NotFound desc = could not find container \"4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f\": container with ID starting with 4cee03301f4d46ecb66dea74616569a670cf67c5b8ddb0d1bd4b295de0e8080f not found: ID does not exist"
Sep 16 17:07:38.382623 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.382588    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:07:38.387313 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:38.387280    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48fbf0-7fbcc969cc-mfs4k"]
Sep 16 17:07:39.398994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:39.398952    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" path="/var/lib/kubelet/pods/a4c986fb-859c-4429-9b8f-2c69cf64ce6c/volumes"
Sep 16 17:07:42.508143 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:07:42.508105    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:08:00.909471 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:00.909435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:08:09.051170 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:09.051120    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:08:14.002864 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:14.002823    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:08:24.070816 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.070772    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" podUID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" containerName="main" containerID="cri-o://d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571" gracePeriod=45
Sep 16 17:08:24.337557 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.337476    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:08:24.465322 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465375    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465473    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nr4r\" (UniqueName: \"kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465752 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache\") pod \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\" (UID: \"63f26443-0f6b-4d93-9eb0-fc998bbb0077\") "
Sep 16 17:08:24.465808 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home" (OuterVolumeSpecName: "home") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:08:24.465876 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:08:24.465968 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.465943    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache" (OuterVolumeSpecName: "model-cache") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:08:24.466203 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.466179    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.466203 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.466203    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.466420 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.466216    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.467831 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.467808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm" (OuterVolumeSpecName: "dshm") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:08:24.467940 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.467855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:08:24.467940 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.467871    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r" (OuterVolumeSpecName: "kube-api-access-4nr4r") pod "63f26443-0f6b-4d93-9eb0-fc998bbb0077" (UID: "63f26443-0f6b-4d93-9eb0-fc998bbb0077"). InnerVolumeSpecName "kube-api-access-4nr4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:08:24.566937 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.566880    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f26443-0f6b-4d93-9eb0-fc998bbb0077-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.566937 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.566931    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f26443-0f6b-4d93-9eb0-fc998bbb0077-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.566937 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.566945    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nr4r\" (UniqueName: \"kubernetes.io/projected/63f26443-0f6b-4d93-9eb0-fc998bbb0077-kube-api-access-4nr4r\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:08:24.620828 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.620723    2577 generic.go:356] "Generic (PLEG): container finished" podID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" containerID="d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571" exitCode=0
Sep 16 17:08:24.620828 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.620770    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"
Sep 16 17:08:24.621058 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.620772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" event={"ID":"63f26443-0f6b-4d93-9eb0-fc998bbb0077","Type":"ContainerDied","Data":"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"}
Sep 16 17:08:24.621058 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.620889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb" event={"ID":"63f26443-0f6b-4d93-9eb0-fc998bbb0077","Type":"ContainerDied","Data":"114bf44ad28a7107bafd7a51ee9087745a221a01c8e5d1e435c2be642b9c9eac"}
Sep 16 17:08:24.621058 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.620909    2577 scope.go:117] "RemoveContainer" containerID="d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"
Sep 16 17:08:24.630673 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.630648    2577 scope.go:117] "RemoveContainer" containerID="d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"
Sep 16 17:08:24.631043 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:08:24.631020    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571\": container with ID starting with d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571 not found: ID does not exist" containerID="d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"
Sep 16 17:08:24.631082 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.631055    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571"} err="failed to get container status \"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571\": rpc error: code = NotFound desc = could not find container \"d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571\": container with ID starting with d8a87cf23dfdff19f1c61d27a2b4572195dfdcdc9b501a44f74dea4e6e9a3571 not found: ID does not exist"
Sep 16 17:08:24.641657 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.641622    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:08:24.646236 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:24.646193    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-9ac315/ipp-route-9ac315-kserve-5f57976c47-jnjkb"]
Sep 16 17:08:25.391400 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:08:25.391357    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" path="/var/lib/kubelet/pods/63f26443-0f6b-4d93-9eb0-fc998bbb0077/volumes"
Sep 16 17:09:17.881425 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:17.881330    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:09:17.881953 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:17.881649    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" podUID="0ef3ac09-241b-4600-b71c-72ca74858ef2" containerName="maas-api" containerID="cri-o://eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc" gracePeriod=30
Sep 16 17:09:18.142081 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.142003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:09:18.239614 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.239565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca\") pod \"0ef3ac09-241b-4600-b71c-72ca74858ef2\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") "
Sep 16 17:09:18.239863 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.239636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs\") pod \"0ef3ac09-241b-4600-b71c-72ca74858ef2\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") "
Sep 16 17:09:18.239863 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.239681    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") pod \"0ef3ac09-241b-4600-b71c-72ca74858ef2\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") "
Sep 16 17:09:18.239863 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.239704    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4j56\" (UniqueName: \"kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56\") pod \"0ef3ac09-241b-4600-b71c-72ca74858ef2\" (UID: \"0ef3ac09-241b-4600-b71c-72ca74858ef2\") "
Sep 16 17:09:18.240088 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.240056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ef3ac09-241b-4600-b71c-72ca74858ef2" (UID: "0ef3ac09-241b-4600-b71c-72ca74858ef2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:18.242230 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.242190    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56" (OuterVolumeSpecName: "kube-api-access-q4j56") pod "0ef3ac09-241b-4600-b71c-72ca74858ef2" (UID: "0ef3ac09-241b-4600-b71c-72ca74858ef2"). InnerVolumeSpecName "kube-api-access-q4j56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:18.242508 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.242487    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ef3ac09-241b-4600-b71c-72ca74858ef2" (UID: "0ef3ac09-241b-4600-b71c-72ca74858ef2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:18.242579 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.242534    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ef3ac09-241b-4600-b71c-72ca74858ef2" (UID: "0ef3ac09-241b-4600-b71c-72ca74858ef2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:18.341586 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.341544    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.341586 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.341576    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4j56\" (UniqueName: \"kubernetes.io/projected/0ef3ac09-241b-4600-b71c-72ca74858ef2-kube-api-access-q4j56\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.341586 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.341586    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef3ac09-241b-4600-b71c-72ca74858ef2-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.341586 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.341595    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3ac09-241b-4600-b71c-72ca74858ef2-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.913941 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.913896    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ef3ac09-241b-4600-b71c-72ca74858ef2" containerID="eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc" exitCode=0
Sep 16 17:09:18.913941 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.913949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"
Sep 16 17:09:18.914512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.913982    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" event={"ID":"0ef3ac09-241b-4600-b71c-72ca74858ef2","Type":"ContainerDied","Data":"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"}
Sep 16 17:09:18.914512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.914044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8" event={"ID":"0ef3ac09-241b-4600-b71c-72ca74858ef2","Type":"ContainerDied","Data":"2ae81766258b296bf6a85b75bee704fe8f92576fd6157c9c645f2660271d3380"}
Sep 16 17:09:18.914512 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.914073    2577 scope.go:117] "RemoveContainer" containerID="eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"
Sep 16 17:09:18.926047 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.926020    2577 scope.go:117] "RemoveContainer" containerID="eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"
Sep 16 17:09:18.926498 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:18.926475    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc\": container with ID starting with eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc not found: ID does not exist" containerID="eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"
Sep 16 17:09:18.926593 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.926510    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc"} err="failed to get container status \"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc\": rpc error: code = NotFound desc = could not find container \"eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc\": container with ID starting with eb88b6ab4a9bfff8fee2904b7137e226169b6e99d6daa437a30eb999237c33dc not found: ID does not exist"
Sep 16 17:09:18.938733 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.938697    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:09:18.942561 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:18.942530    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6586c3-6877dbbcc8-49fc8"]
Sep 16 17:09:19.391414 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:19.391336    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ef3ac09-241b-4600-b71c-72ca74858ef2" path="/var/lib/kubelet/pods/0ef3ac09-241b-4600-b71c-72ca74858ef2/volumes"
Sep 16 17:09:21.632991 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:21.632950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:09:24.541580 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:24.541538    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:09:24.542054 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:24.541796    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" containerID="cri-o://d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c" gracePeriod=30
Sep 16 17:09:27.624630 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:27.624590    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 17:09:27.625062 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:27.624659    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:09:29.804169 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.804097    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:09:29.956650 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9kmd\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.956841 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956690    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.956841 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956718    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.956841 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.956841 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.957108 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956845    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.957108 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956879    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.957108 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.957108 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.956976    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data\") pod \"dd757286-0a59-465b-b7c5-8670a5f60153\" (UID: \"dd757286-0a59-465b-b7c5-8670a5f60153\") "
Sep 16 17:09:29.957335 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:09:29.957443 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957342    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:29.957513 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957465    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:09:29.957574 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:09:29.957632 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957343    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:29.957690 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.957663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:09:29.959172 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.959143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:09:29.959172 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.959161    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd" (OuterVolumeSpecName: "kube-api-access-c9kmd") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "kube-api-access-c9kmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:29.959473 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.959441    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:09:29.959473 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.959450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd757286-0a59-465b-b7c5-8670a5f60153" (UID: "dd757286-0a59-465b-b7c5-8670a5f60153"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:29.976339 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.976302    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd757286-0a59-465b-b7c5-8670a5f60153" containerID="d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c" exitCode=0
Sep 16 17:09:29.976537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.976363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"
Sep 16 17:09:29.976537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.976406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerDied","Data":"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"}
Sep 16 17:09:29.976537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.976474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff" event={"ID":"dd757286-0a59-465b-b7c5-8670a5f60153","Type":"ContainerDied","Data":"3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368"}
Sep 16 17:09:29.976537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.976504    2577 scope.go:117] "RemoveContainer" containerID="d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"
Sep 16 17:09:29.985969 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.985947    2577 scope.go:117] "RemoveContainer" containerID="a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"
Sep 16 17:09:29.998005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.997981    2577 scope.go:117] "RemoveContainer" containerID="d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"
Sep 16 17:09:29.998347 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:29.998325    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c\": container with ID starting with d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c not found: ID does not exist" containerID="d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"
Sep 16 17:09:29.998450 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.998356    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c"} err="failed to get container status \"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c\": rpc error: code = NotFound desc = could not find container \"d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c\": container with ID starting with d04ae3f1fcf0ab6d882cfdd93cf4b3d5c5642b71d10c3fff44b4c40e8dd1bd3c not found: ID does not exist"
Sep 16 17:09:29.998450 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.998374    2577 scope.go:117] "RemoveContainer" containerID="a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"
Sep 16 17:09:29.998678 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:29.998657    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f\": container with ID starting with a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f not found: ID does not exist" containerID="a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"
Sep 16 17:09:29.998749 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:29.998688    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f"} err="failed to get container status \"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f\": rpc error: code = NotFound desc = could not find container \"a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f\": container with ID starting with a765f7fe49729cbaed74e41a05fbfd698dcb850de1f284371d9b11d777fda64f not found: ID does not exist"
Sep 16 17:09:30.002136 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.002107    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:09:30.007436 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.007409    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6586c3-openshift-default-6698875954-9wjff"]
Sep 16 17:09:30.026251 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:30.026212    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd757286_0a59_465b_b7c5_8670a5f60153.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd757286_0a59_465b_b7c5_8670a5f60153.slice/crio-3508df60c5e6290de752d9cdf14caaa599c691286336f24eddd4711af754d368\": RecentStats: unable to find data in memory cache]"
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.057952    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.057986    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.057997    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.058029    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9kmd\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-kube-api-access-c9kmd\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.058040    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd757286-0a59-465b-b7c5-8670a5f60153-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058041 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.058048    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd757286-0a59-465b-b7c5-8670a5f60153-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058344 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.058056    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd757286-0a59-465b-b7c5-8670a5f60153-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:30.058344 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:30.058063    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd757286-0a59-465b-b7c5-8670a5f60153-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:31.390960 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:31.390919    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" path="/var/lib/kubelet/pods/dd757286-0a59-465b-b7c5-8670a5f60153/volumes"
Sep 16 17:09:35.328341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.328296    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 17:09:35.329017 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.328586    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" podUID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" containerName="maas-api" containerID="cri-o://94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75" gracePeriod=30
Sep 16 17:09:35.591231 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.591154    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 17:09:35.711645 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.711605    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs\") pod \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") "
Sep 16 17:09:35.711839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.711665    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca\") pod \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") "
Sep 16 17:09:35.711839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.711735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r75gk\" (UniqueName: \"kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk\") pod \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") "
Sep 16 17:09:35.711839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.711777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls\") pod \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\" (UID: \"0ac9100e-638f-4949-aa80-ea6167a8d3d9\") "
Sep 16 17:09:35.712068 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.712038    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ac9100e-638f-4949-aa80-ea6167a8d3d9" (UID: "0ac9100e-638f-4949-aa80-ea6167a8d3d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:35.712241 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.712213    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9100e-638f-4949-aa80-ea6167a8d3d9-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.714030 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.713997    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ac9100e-638f-4949-aa80-ea6167a8d3d9" (UID: "0ac9100e-638f-4949-aa80-ea6167a8d3d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:35.714139 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.714025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ac9100e-638f-4949-aa80-ea6167a8d3d9" (UID: "0ac9100e-638f-4949-aa80-ea6167a8d3d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:35.714139 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.714003    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk" (OuterVolumeSpecName: "kube-api-access-r75gk") pod "0ac9100e-638f-4949-aa80-ea6167a8d3d9" (UID: "0ac9100e-638f-4949-aa80-ea6167a8d3d9"). InnerVolumeSpecName "kube-api-access-r75gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:35.812839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.812798    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r75gk\" (UniqueName: \"kubernetes.io/projected/0ac9100e-638f-4949-aa80-ea6167a8d3d9-kube-api-access-r75gk\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.812839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.812834    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.812839 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:35.812843    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac9100e-638f-4949-aa80-ea6167a8d3d9-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:36.010249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.010151    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" containerID="94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75" exitCode=0
Sep 16 17:09:36.010249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.010195    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"
Sep 16 17:09:36.010249 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.010236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" event={"ID":"0ac9100e-638f-4949-aa80-ea6167a8d3d9","Type":"ContainerDied","Data":"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"}
Sep 16 17:09:36.010546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.010296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq" event={"ID":"0ac9100e-638f-4949-aa80-ea6167a8d3d9","Type":"ContainerDied","Data":"aeeba2fd952e634c8aaa94da95f26c3e7710cee873a3729404300cad7d5dec45"}
Sep 16 17:09:36.010546 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.010320    2577 scope.go:117] "RemoveContainer" containerID="94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"
Sep 16 17:09:36.020478 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.020456    2577 scope.go:117] "RemoveContainer" containerID="94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"
Sep 16 17:09:36.020833 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:36.020811    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75\": container with ID starting with 94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75 not found: ID does not exist" containerID="94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"
Sep 16 17:09:36.020904 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.020843    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75"} err="failed to get container status \"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75\": rpc error: code = NotFound desc = could not find container \"94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75\": container with ID starting with 94e0cdafde33c05ed895e05fddfeb3a949d8d0c1e7b2ed89047a6dced011ad75 not found: ID does not exist"
Sep 16 17:09:36.043628 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.043599    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 17:09:36.061137 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:36.061099    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-52fd8c-685d985bd9-cjzsq"]
Sep 16 17:09:37.392643 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:37.392604    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" path="/var/lib/kubelet/pods/0ac9100e-638f-4949-aa80-ea6167a8d3d9/volumes"
Sep 16 17:09:38.406613 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:38.406573    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:09:53.535278 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.535241    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 17:09:53.535746 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.535492    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" podUID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" containerName="maas-api" containerID="cri-o://dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d" gracePeriod=30
Sep 16 17:09:53.791073 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.791042    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 17:09:53.885970 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.885937    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs\") pod \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") "
Sep 16 17:09:53.886185 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.885988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca\") pod \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") "
Sep 16 17:09:53.886185 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.886012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") pod \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") "
Sep 16 17:09:53.886185 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.886028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8mx9\" (UniqueName: \"kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9\") pod \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\" (UID: \"d4f6da88-3dc3-4793-8fdb-b33a73d05309\") "
Sep 16 17:09:53.886522 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.886499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4f6da88-3dc3-4793-8fdb-b33a73d05309" (UID: "d4f6da88-3dc3-4793-8fdb-b33a73d05309"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:53.888330 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.888299    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4f6da88-3dc3-4793-8fdb-b33a73d05309" (UID: "d4f6da88-3dc3-4793-8fdb-b33a73d05309"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:53.888449 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.888312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9" (OuterVolumeSpecName: "kube-api-access-r8mx9") pod "d4f6da88-3dc3-4793-8fdb-b33a73d05309" (UID: "d4f6da88-3dc3-4793-8fdb-b33a73d05309"). InnerVolumeSpecName "kube-api-access-r8mx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:53.888449 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.888333    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d4f6da88-3dc3-4793-8fdb-b33a73d05309" (UID: "d4f6da88-3dc3-4793-8fdb-b33a73d05309"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:09:53.987756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.987715    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:53.987756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.987748    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f6da88-3dc3-4793-8fdb-b33a73d05309-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:53.987756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.987761    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4f6da88-3dc3-4793-8fdb-b33a73d05309-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:53.988026 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:53.987774    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8mx9\" (UniqueName: \"kubernetes.io/projected/d4f6da88-3dc3-4793-8fdb-b33a73d05309-kube-api-access-r8mx9\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:09:54.107325 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.107227    2577 generic.go:356] "Generic (PLEG): container finished" podID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" containerID="dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d" exitCode=0
Sep 16 17:09:54.107325 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.107271    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"
Sep 16 17:09:54.107592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.107319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" event={"ID":"d4f6da88-3dc3-4793-8fdb-b33a73d05309","Type":"ContainerDied","Data":"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"}
Sep 16 17:09:54.107592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.107414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz" event={"ID":"d4f6da88-3dc3-4793-8fdb-b33a73d05309","Type":"ContainerDied","Data":"abc83cca99bd2497c33131a2873fd1d6e3aad1218ddbc531aad67ca6132fa1a9"}
Sep 16 17:09:54.107592 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.107448    2577 scope.go:117] "RemoveContainer" containerID="dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"
Sep 16 17:09:54.117751 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.117729    2577 scope.go:117] "RemoveContainer" containerID="dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"
Sep 16 17:09:54.118077 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:09:54.118056    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d\": container with ID starting with dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d not found: ID does not exist" containerID="dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"
Sep 16 17:09:54.118129 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.118087    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d"} err="failed to get container status \"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d\": rpc error: code = NotFound desc = could not find container \"dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d\": container with ID starting with dbccb3db79c80a7d09b55e915a63dcbce04a51d146f957a62ea26303129cd54d not found: ID does not exist"
Sep 16 17:09:54.129589 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.129556    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 17:09:54.132752 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:54.132726    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34d301-65456654c7-8gnjz"]
Sep 16 17:09:55.391010 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:55.390974    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" path="/var/lib/kubelet/pods/d4f6da88-3dc3-4793-8fdb-b33a73d05309/volumes"
Sep 16 17:09:56.200582 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:09:56.200540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:10:00.649137 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:00.649102    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 17:10:00.649670 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:00.649369    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy" containerID="cri-o://67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3" gracePeriod=30
Sep 16 17:10:04.906514 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:04.906472    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:10:05.925780 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:05.925690    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 17:10:06.005574 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005528    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.005574 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005573    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.005799 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.005799 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.005799 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.005799 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.006003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005821    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djlm6\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.006003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005862    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.006003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005871    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:10:06.006003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:10:06.006003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.005934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo\") pod \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\" (UID: \"9cf46f4a-5d20-4b97-b45f-300886bd5efd\") "
Sep 16 17:10:06.006261 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:10:06.006261 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006214    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:10:06.006361 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006324    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:10:06.006462 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006425    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-credential-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.006462 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006448    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-socket\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.006569 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006463    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istiod-ca-cert\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.006569 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.006478    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-workload-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.008197 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.008166    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6" (OuterVolumeSpecName: "kube-api-access-djlm6") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "kube-api-access-djlm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:10:06.008344 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.008205    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:10:06.008344 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.008243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:10:06.008344 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.008255    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cf46f4a-5d20-4b97-b45f-300886bd5efd" (UID: "9cf46f4a-5d20-4b97-b45f-300886bd5efd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:10:06.107936 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.107895    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-data\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.107936 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.107928    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djlm6\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-kube-api-access-djlm6\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.107936 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.107939    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-token\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.107936 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.107948    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-podinfo\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.108222 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.107957    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf46f4a-5d20-4b97-b45f-300886bd5efd-istio-envoy\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:10:06.171341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.171305    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerID="67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3" exitCode=0
Sep 16 17:10:06.171341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.171347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerDied","Data":"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"}
Sep 16 17:10:06.171650 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.171363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"
Sep 16 17:10:06.171650 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.171405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5" event={"ID":"9cf46f4a-5d20-4b97-b45f-300886bd5efd","Type":"ContainerDied","Data":"314eab65f8f2529e9dda27bf5fb040dc6dae0375eb8a8740ed72f60121684f26"}
Sep 16 17:10:06.171650 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.171427    2577 scope.go:117] "RemoveContainer" containerID="67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"
Sep 16 17:10:06.182191 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.182166    2577 scope.go:117] "RemoveContainer" containerID="6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"
Sep 16 17:10:06.191453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.191429    2577 scope.go:117] "RemoveContainer" containerID="67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"
Sep 16 17:10:06.191838 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:10:06.191809    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3\": container with ID starting with 67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3 not found: ID does not exist" containerID="67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"
Sep 16 17:10:06.191961 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.191850    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3"} err="failed to get container status \"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3\": rpc error: code = NotFound desc = could not find container \"67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3\": container with ID starting with 67079c2c601998f4127ff2b22387c49f6df0e6c71941f727bdecea75db4bd3f3 not found: ID does not exist"
Sep 16 17:10:06.191961 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.191875    2577 scope.go:117] "RemoveContainer" containerID="6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"
Sep 16 17:10:06.192206 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:10:06.192189    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc\": container with ID starting with 6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc not found: ID does not exist" containerID="6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"
Sep 16 17:10:06.192246 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.192213    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc"} err="failed to get container status \"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc\": rpc error: code = NotFound desc = could not find container \"6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc\": container with ID starting with 6e2e4caa9951cc6325403e80c9832f1a7e7f9a62dfbd43397bd2fc12d95339dc not found: ID does not exist"
Sep 16 17:10:06.193034 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.193013    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 17:10:06.197307 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:06.197281    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34d301-openshift-default-cdcb745fb-mwsd5"]
Sep 16 17:10:07.391947 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:10:07.391911    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" path="/var/lib/kubelet/pods/9cf46f4a-5d20-4b97-b45f-300886bd5efd/volumes"
Sep 16 17:11:28.803719 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:11:28.803677    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:11:54.495629 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:11:54.495589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:12:02.507373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:12:02.507336    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:12:10.915240 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:12:10.915198    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:12:28.205685 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:12:28.205593    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:12:34.108621 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:12:34.108585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:14:04.332069 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:04.332031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:14:10.309130 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:10.309090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:14:37.921067 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921451    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" containerName="main"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921466    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" containerName="main"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921476    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921482    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921488    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921493    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921513    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921521    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921527    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921542    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921547    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921559    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ef3ac09-241b-4600-b71c-72ca74858ef2" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921567    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ef3ac09-241b-4600-b71c-72ca74858ef2" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921632    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f6da88-3dc3-4793-8fdb-b33a73d05309" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921643    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac9100e-638f-4949-aa80-ea6167a8d3d9" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921650    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ef3ac09-241b-4600-b71c-72ca74858ef2" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921655    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="63f26443-0f6b-4d93-9eb0-fc998bbb0077" containerName="main"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921661    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921667    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c986fb-859c-4429-9b8f-2c69cf64ce6c" containerName="maas-api"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921672    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:14:37.921704 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.921677    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:14:37.924708 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.924680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:37.927186 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.927167    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8c71e597-kserve-self-signed-certs\""
Sep 16 17:14:37.941563 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:37.941530    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:14:38.013135 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7v2\" (UniqueName: \"kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.013135 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.013415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.013415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.013415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.013415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.013324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114483 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7v2\" (UniqueName: \"kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114483 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114755 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114755 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114755 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.114755 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.115005 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.114983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.115073 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.115017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.115073 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.115047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.117078 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.117051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.117325 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.117306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.123485 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.123460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7v2\" (UniqueName: \"kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2\") pod \"e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") " pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.234280 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.234180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:38.511106 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.510911    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:14:38.514042 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:14:38.514012    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebf835f6_358a_4d40_8d31_220987a1fb46.slice/crio-5d156f1cce9d5ebba1557c26f409c3b9c18a8cf178914dbd7d9ee1962142e6d0 WatchSource:0}: Error finding container 5d156f1cce9d5ebba1557c26f409c3b9c18a8cf178914dbd7d9ee1962142e6d0: Status 404 returned error can't find the container with id 5d156f1cce9d5ebba1557c26f409c3b9c18a8cf178914dbd7d9ee1962142e6d0
Sep 16 17:14:38.580956 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:38.580916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" event={"ID":"ebf835f6-358a-4d40-8d31-220987a1fb46","Type":"ContainerStarted","Data":"5d156f1cce9d5ebba1557c26f409c3b9c18a8cf178914dbd7d9ee1962142e6d0"}
Sep 16 17:14:39.587766 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:39.587727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" event={"ID":"ebf835f6-358a-4d40-8d31-220987a1fb46","Type":"ContainerStarted","Data":"5d672375a2f393cd1ce9a5cc46095df441a3e7cccf79143168b024fd58219fbf"}
Sep 16 17:14:39.612333 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:39.612275    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" podStartSLOduration=2.6122588860000002 podStartE2EDuration="2.612258886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:14:39.607890564 +0000 UTC m=+2296.693610635" watchObservedRunningTime="2026-09-16 17:14:39.612258886 +0000 UTC m=+2296.697978957"
Sep 16 17:14:40.520003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:40.519969    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:14:46.991243 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:46.991208    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:14:48.234519 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:48.234462    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:48.234519 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:48.234532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:48.247590 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:48.247551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:48.649177 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:48.649136    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:14:59.528404 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:14:59.528358    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:15:00.139817 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.139778    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"]
Sep 16 17:15:00.140193 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.140180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:15:00.140245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.140194    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf46f4a-5d20-4b97-b45f-300886bd5efd" containerName="istio-proxy"
Sep 16 17:15:00.140245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.140204    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:15:00.140245 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.140210    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:15:00.140345 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.140283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd757286-0a59-465b-b7c5-8670a5f60153" containerName="istio-proxy"
Sep 16 17:15:00.143289 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.143269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"
Sep 16 17:15:00.145661 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.145644    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:15:00.150502 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.150470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"]
Sep 16 17:15:00.156625 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.156594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"
Sep 16 17:15:00.430923 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.430838    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"]
Sep 16 17:15:00.434763 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:15:00.434733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9aa890_1cff_4172_902b_97273c1efe66.slice/crio-b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c WatchSource:0}: Error finding container b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c: Status 404 returned error can't find the container with id b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c
Sep 16 17:15:00.697060 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:00.696973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk" event={"ID":"8e9aa890-1cff-4172-902b-97273c1efe66","Type":"ContainerStarted","Data":"b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c"}
Sep 16 17:15:01.704425 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:01.704302    2577 generic.go:356] "Generic (PLEG): container finished" podID="8e9aa890-1cff-4172-902b-97273c1efe66" containerID="7a79685955888abc2576a12e78100b3e17f06928e6165c1910fea94495faa7e2" exitCode=0
Sep 16 17:15:01.704425 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:01.704345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk" event={"ID":"8e9aa890-1cff-4172-902b-97273c1efe66","Type":"ContainerDied","Data":"7a79685955888abc2576a12e78100b3e17f06928e6165c1910fea94495faa7e2"}
Sep 16 17:15:02.707511 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:02.707468    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:02.854766 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:02.854739    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"
Sep 16 17:15:03.717050 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:03.716960    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk"
Sep 16 17:15:03.717050 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:03.716954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826315-j49dk" event={"ID":"8e9aa890-1cff-4172-902b-97273c1efe66","Type":"ContainerDied","Data":"b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c"}
Sep 16 17:15:03.717567 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:03.717092    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b62a49f9226643b0c69e5b91ae2ac1c93c76a3fe33bda1a856f062eb2b229e1c"
Sep 16 17:15:11.614984 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:11.614945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:14.548047 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.547973    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" podUID="ebf835f6-358a-4d40-8d31-220987a1fb46" containerName="main" containerID="cri-o://5d672375a2f393cd1ce9a5cc46095df441a3e7cccf79143168b024fd58219fbf" gracePeriod=45
Sep 16 17:15:14.779172 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.779132    2577 generic.go:356] "Generic (PLEG): container finished" podID="ebf835f6-358a-4d40-8d31-220987a1fb46" containerID="5d672375a2f393cd1ce9a5cc46095df441a3e7cccf79143168b024fd58219fbf" exitCode=0
Sep 16 17:15:14.779350 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.779211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" event={"ID":"ebf835f6-358a-4d40-8d31-220987a1fb46","Type":"ContainerDied","Data":"5d672375a2f393cd1ce9a5cc46095df441a3e7cccf79143168b024fd58219fbf"}
Sep 16 17:15:14.805341 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.805271    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:15:14.878143 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm7v2\" (UniqueName: \"kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878213    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878334    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs\") pod \"ebf835f6-358a-4d40-8d31-220987a1fb46\" (UID: \"ebf835f6-358a-4d40-8d31-220987a1fb46\") "
Sep 16 17:15:14.878635 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878523    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home" (OuterVolumeSpecName: "home") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:15:14.878635 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878616    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache" (OuterVolumeSpecName: "model-cache") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:15:14.878753 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:15:14.878801 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878751    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-home\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:14.878801 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878764    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-model-cache\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:14.878801 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.878775    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-tmp-dir\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:14.880618 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.880595    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm" (OuterVolumeSpecName: "dshm") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:15:14.880700 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.880623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:15:14.880700 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.880679    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2" (OuterVolumeSpecName: "kube-api-access-gm7v2") pod "ebf835f6-358a-4d40-8d31-220987a1fb46" (UID: "ebf835f6-358a-4d40-8d31-220987a1fb46"). InnerVolumeSpecName "kube-api-access-gm7v2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:15:14.980352 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.980315    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm7v2\" (UniqueName: \"kubernetes.io/projected/ebf835f6-358a-4d40-8d31-220987a1fb46-kube-api-access-gm7v2\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:14.980352 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.980343    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf835f6-358a-4d40-8d31-220987a1fb46-tls-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:14.980352 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:14.980356    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf835f6-358a-4d40-8d31-220987a1fb46-dshm\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:15.785817 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:15.785728    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"
Sep 16 17:15:15.785817 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:15.785765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j" event={"ID":"ebf835f6-358a-4d40-8d31-220987a1fb46","Type":"ContainerDied","Data":"5d156f1cce9d5ebba1557c26f409c3b9c18a8cf178914dbd7d9ee1962142e6d0"}
Sep 16 17:15:15.785817 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:15.785815    2577 scope.go:117] "RemoveContainer" containerID="5d672375a2f393cd1ce9a5cc46095df441a3e7cccf79143168b024fd58219fbf"
Sep 16 17:15:15.802994 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:15.802955    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:15:15.806378 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:15.806347    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8c71e597-kserve-7f47f48c8c-lsv4j"]
Sep 16 17:15:17.391220 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:17.391185    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf835f6-358a-4d40-8d31-220987a1fb46" path="/var/lib/kubelet/pods/ebf835f6-358a-4d40-8d31-220987a1fb46/volumes"
Sep 16 17:15:20.911990 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:20.911949    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:21.350863 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.350819    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:21.351242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351228    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e9aa890-1cff-4172-902b-97273c1efe66" containerName="cleanup"
Sep 16 17:15:21.351242 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351243    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e9aa890-1cff-4172-902b-97273c1efe66" containerName="cleanup"
Sep 16 17:15:21.351373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351263    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf835f6-358a-4d40-8d31-220987a1fb46" containerName="main"
Sep 16 17:15:21.351373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351269    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf835f6-358a-4d40-8d31-220987a1fb46" containerName="main"
Sep 16 17:15:21.351373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351335    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf835f6-358a-4d40-8d31-220987a1fb46" containerName="main"
Sep 16 17:15:21.351373 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.351342    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e9aa890-1cff-4172-902b-97273c1efe66" containerName="cleanup"
Sep 16 17:15:21.356050 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.356022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.359215 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.359182    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 17:15:21.359429 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.359222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qc55\""
Sep 16 17:15:21.359429 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.359234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 17:15:21.359429 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.359187    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:15:21.359429 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.359182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 17:15:21.367407 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.367354    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:21.443601 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.443560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.443794 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.443680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.443794 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.443768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj7nj\" (UniqueName: \"kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.443877 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.443813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.544744 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.544706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.544973 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.544796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.544973 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.544853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj7nj\" (UniqueName: \"kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.544973 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.544887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.545715 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.545685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.547329 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.547298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.547604 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.547581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.552757 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.552730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj7nj\" (UniqueName: \"kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj\") pod \"maas-api-5665c7f4f4-85r7v\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") " pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.669022 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.668910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:21.943415 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.943368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:21.945984 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:15:21.945953    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42511b19_a03b_4459_9c9d_6a7d7dc2928b.slice/crio-1df5118784dbb9facd2a7b897e5c18ffa05c77e2f82b41392b083ab0613f853d WatchSource:0}: Error finding container 1df5118784dbb9facd2a7b897e5c18ffa05c77e2f82b41392b083ab0613f853d: Status 404 returned error can't find the container with id 1df5118784dbb9facd2a7b897e5c18ffa05c77e2f82b41392b083ab0613f853d
Sep 16 17:15:21.947773 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:21.947746    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:15:22.824003 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:22.823965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" event={"ID":"42511b19-a03b-4459-9c9d-6a7d7dc2928b","Type":"ContainerStarted","Data":"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"}
Sep 16 17:15:22.824235 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:22.824027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" event={"ID":"42511b19-a03b-4459-9c9d-6a7d7dc2928b","Type":"ContainerStarted","Data":"1df5118784dbb9facd2a7b897e5c18ffa05c77e2f82b41392b083ab0613f853d"}
Sep 16 17:15:22.824235 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:22.824056    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:22.842217 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:22.842159    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" podStartSLOduration=1.458100801 podStartE2EDuration="1.842142299s" totalImagesPullingTime="384.041498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:15:22.840012611 +0000 UTC m=+2339.925732687" watchObservedRunningTime="2026-09-16 17:15:22.842142299 +0000 UTC m=+2339.927862371"
Sep 16 17:15:28.700256 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:28.700213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:28.834710 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:28.834679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:29.553405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.553356    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"]
Sep 16 17:15:29.559020 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.558989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.571919 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.571881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"]
Sep 16 17:15:29.636145 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.636100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-metrics-certs\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.636427 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.636175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-maas-api-tls\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.636427 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.636242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw2gm\" (UniqueName: \"kubernetes.io/projected/2592746a-889e-4317-82f3-d068bf201877-kube-api-access-qw2gm\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.636427 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.636310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2592746a-889e-4317-82f3-d068bf201877-openshift-service-ca\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.737445 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.737376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2592746a-889e-4317-82f3-d068bf201877-openshift-service-ca\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.737981 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.737504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-metrics-certs\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.737981 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.737579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-maas-api-tls\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.737981 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.737601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw2gm\" (UniqueName: \"kubernetes.io/projected/2592746a-889e-4317-82f3-d068bf201877-kube-api-access-qw2gm\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.738276 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.738238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2592746a-889e-4317-82f3-d068bf201877-openshift-service-ca\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.740578 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.740543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-maas-api-tls\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.740743 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.740698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2592746a-889e-4317-82f3-d068bf201877-metrics-certs\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.746719 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.746681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw2gm\" (UniqueName: \"kubernetes.io/projected/2592746a-889e-4317-82f3-d068bf201877-kube-api-access-qw2gm\") pod \"maas-api-5449d58978-fpfvl\" (UID: \"2592746a-889e-4317-82f3-d068bf201877\") " pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:29.876084 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:29.875984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:30.154867 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:30.154831    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"]
Sep 16 17:15:30.157745 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:15:30.157711    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2592746a_889e_4317_82f3_d068bf201877.slice/crio-4148032f4cc5f10054893465507b129dd2a934933ad40f3fb4bdcd64383b0961 WatchSource:0}: Error finding container 4148032f4cc5f10054893465507b129dd2a934933ad40f3fb4bdcd64383b0961: Status 404 returned error can't find the container with id 4148032f4cc5f10054893465507b129dd2a934933ad40f3fb4bdcd64383b0961
Sep 16 17:15:30.869620 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:30.869524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl" event={"ID":"2592746a-889e-4317-82f3-d068bf201877","Type":"ContainerStarted","Data":"d95421bdd077ef8e3d02762d0e56d4c5ebfec6c546bedd33d400b202a4c8240c"}
Sep 16 17:15:30.869620 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:30.869577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl" event={"ID":"2592746a-889e-4317-82f3-d068bf201877","Type":"ContainerStarted","Data":"4148032f4cc5f10054893465507b129dd2a934933ad40f3fb4bdcd64383b0961"}
Sep 16 17:15:30.870083 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:30.869663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:30.887870 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:30.887816    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl" podStartSLOduration=1.455949683 podStartE2EDuration="1.887797102s" totalImagesPullingTime="431.847419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:15:30.886994037 +0000 UTC m=+2347.972714111" watchObservedRunningTime="2026-09-16 17:15:30.887797102 +0000 UTC m=+2347.973517173"
Sep 16 17:15:35.406534 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:35.406486    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:36.880441 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:36.880408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5449d58978-fpfvl"
Sep 16 17:15:36.928233 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:36.928198    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:36.928504 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:36.928467    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" podUID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" containerName="maas-api" containerID="cri-o://d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c" gracePeriod=30
Sep 16 17:15:38.192987 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.192957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:38.325122 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325083    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs\") pod \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") "
Sep 16 17:15:38.325453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls\") pod \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") "
Sep 16 17:15:38.325453 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325327    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca\") pod \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") "
Sep 16 17:15:38.325605 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj7nj\" (UniqueName: \"kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj\") pod \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\" (UID: \"42511b19-a03b-4459-9c9d-6a7d7dc2928b\") "
Sep 16 17:15:38.325676 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42511b19-a03b-4459-9c9d-6a7d7dc2928b" (UID: "42511b19-a03b-4459-9c9d-6a7d7dc2928b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:15:38.325790 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.325776    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42511b19-a03b-4459-9c9d-6a7d7dc2928b-openshift-service-ca\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:38.327429 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.327370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "42511b19-a03b-4459-9c9d-6a7d7dc2928b" (UID: "42511b19-a03b-4459-9c9d-6a7d7dc2928b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:15:38.327552 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.327433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42511b19-a03b-4459-9c9d-6a7d7dc2928b" (UID: "42511b19-a03b-4459-9c9d-6a7d7dc2928b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:15:38.327633 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.327616    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj" (OuterVolumeSpecName: "kube-api-access-lj7nj") pod "42511b19-a03b-4459-9c9d-6a7d7dc2928b" (UID: "42511b19-a03b-4459-9c9d-6a7d7dc2928b"). InnerVolumeSpecName "kube-api-access-lj7nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:15:38.426510 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.426473    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj7nj\" (UniqueName: \"kubernetes.io/projected/42511b19-a03b-4459-9c9d-6a7d7dc2928b-kube-api-access-lj7nj\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:38.426510 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.426506    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-metrics-certs\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:38.426510 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.426515    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42511b19-a03b-4459-9c9d-6a7d7dc2928b-maas-api-tls\") on node \"ip-10-0-132-17.ec2.internal\" DevicePath \"\""
Sep 16 17:15:38.919114 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.919079    2577 generic.go:356] "Generic (PLEG): container finished" podID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" containerID="d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c" exitCode=0
Sep 16 17:15:38.919313 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.919132    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"
Sep 16 17:15:38.919313 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.919164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" event={"ID":"42511b19-a03b-4459-9c9d-6a7d7dc2928b","Type":"ContainerDied","Data":"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"}
Sep 16 17:15:38.919313 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.919251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v" event={"ID":"42511b19-a03b-4459-9c9d-6a7d7dc2928b","Type":"ContainerDied","Data":"1df5118784dbb9facd2a7b897e5c18ffa05c77e2f82b41392b083ab0613f853d"}
Sep 16 17:15:38.919313 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.919275    2577 scope.go:117] "RemoveContainer" containerID="d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"
Sep 16 17:15:38.930375 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.930352    2577 scope.go:117] "RemoveContainer" containerID="d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"
Sep 16 17:15:38.930759 ip-10-0-132-17 kubenswrapper[2577]: E0916 17:15:38.930735    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c\": container with ID starting with d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c not found: ID does not exist" containerID="d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"
Sep 16 17:15:38.930877 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.930769    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c"} err="failed to get container status \"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c\": rpc error: code = NotFound desc = could not find container \"d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c\": container with ID starting with d52284a02978f15e53c69ea6c20d011ff77289dbcec34831749b411aaa28542c not found: ID does not exist"
Sep 16 17:15:38.944400 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.944349    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:38.950110 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:38.950077    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5665c7f4f4-85r7v"]
Sep 16 17:15:39.391873 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:39.391836    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" path="/var/lib/kubelet/pods/42511b19-a03b-4459-9c9d-6a7d7dc2928b/volumes"
Sep 16 17:15:41.313470 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:41.313430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:48.120354 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:48.120314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-88ttm"]
Sep 16 17:15:55.902673 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:55.902639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4xkbz_f3a1c5c9-5019-44bf-8103-3b285fa9893e/manager/0.log"
Sep 16 17:15:56.021522 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:56.021481    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zs5g5_43095e77-16dc-4cec-a9fd-a684496902ed/manager/0.log"
Sep 16 17:15:56.730027 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:56.729997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-k699z_5ab49632-219f-48f6-9dc2-ae10a6602707/copy-manifests/0.log"
Sep 16 17:15:56.736400 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:56.736360    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-k699z_5ab49632-219f-48f6-9dc2-ae10a6602707/manager/0.log"
Sep 16 17:15:57.067522 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:57.067495    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5449d58978-fpfvl_2592746a-889e-4317-82f3-d068bf201877/maas-api/0.log"
Sep 16 17:15:57.279261 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:57.279232    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826300-hnw5b_eb8ac18d-681d-44c1-9dd3-d9be8f9cc408/cleanup/0.log"
Sep 16 17:15:57.392975 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:57.392874    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826315-j49dk_8e9aa890-1cff-4172-902b-97273c1efe66/cleanup/0.log"
Sep 16 17:15:58.490845 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:58.490813    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-gvzmp_56db9976-26c8-40da-9745-d08b0b8b3df8/manager/0.log"
Sep 16 17:15:58.607159 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:58.607130    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-z6t9j_579f7a0c-5ff4-44cd-874e-fac14378a226/manager/0.log"
Sep 16 17:15:58.978665 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:58.978634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-88ttm_3ff645b4-76c6-48ab-84be-319f7b8ba8e0/limitador/0.log"
Sep 16 17:15:59.438819 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:59.438788    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2_dbb947a5-3b59-40e6-8efe-3ede5e7bd29d/istio-proxy/0.log"
Sep 16 17:15:59.890237 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:15:59.890204    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/1.log"
Sep 16 17:16:00.681795 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:00.681747    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-czj4z_5c5c92ff-5e97-4859-995f-b7f57ee97940/main/0.log"
Sep 16 17:16:01.218536 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:01.218504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-jzcxd_e2a270cf-6aa7-434e-9c3a-0e9436eda014/main/0.log"
Sep 16 17:16:10.176247 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:10.176211    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-56zhm_dcf2eb2e-913a-4d2a-9b7b-f82bc24971ad/global-pull-secret-syncer/0.log"
Sep 16 17:16:10.417927 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:10.417876    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t8fd8_6a55152b-766f-402d-a86f-ad31b37d9e66/konnectivity-agent/0.log"
Sep 16 17:16:10.483138 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:10.483050    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-17.ec2.internal_6cba30f3283a64fbc23b4dd01a869002/haproxy/0.log"
Sep 16 17:16:14.855181 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:14.855147    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-gvzmp_56db9976-26c8-40da-9745-d08b0b8b3df8/manager/0.log"
Sep 16 17:16:14.886338 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:14.886302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-z6t9j_579f7a0c-5ff4-44cd-874e-fac14378a226/manager/0.log"
Sep 16 17:16:15.001462 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:15.001418    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-88ttm_3ff645b4-76c6-48ab-84be-319f7b8ba8e0/limitador/0.log"
Sep 16 17:16:16.817166 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:16.817135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2kn8d_cd87b230-c3f0-4fb0-bd3e-246153747821/kube-state-metrics/0.log"
Sep 16 17:16:16.841434 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:16.841376    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2kn8d_cd87b230-c3f0-4fb0-bd3e-246153747821/kube-rbac-proxy-main/0.log"
Sep 16 17:16:16.867616 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:16.867585    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2kn8d_cd87b230-c3f0-4fb0-bd3e-246153747821/kube-rbac-proxy-self/0.log"
Sep 16 17:16:16.891786 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:16.891682    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5dc4665f77-5nhqh_1fd729a5-2884-47d9-a384-f5dd56f221f3/metrics-server/0.log"
Sep 16 17:16:17.061535 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.061507    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-677qz_b729877b-df32-46d1-8389-5fd69da6ed1e/node-exporter/0.log"
Sep 16 17:16:17.090520 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.090490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-677qz_b729877b-df32-46d1-8389-5fd69da6ed1e/kube-rbac-proxy/0.log"
Sep 16 17:16:17.126662 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.126638    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-677qz_b729877b-df32-46d1-8389-5fd69da6ed1e/init-textfile/0.log"
Sep 16 17:16:17.275230 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.275199    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-c88t6_cc8a086f-7f07-4325-a25f-99b769b8ca0f/kube-rbac-proxy-main/0.log"
Sep 16 17:16:17.304547 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.304515    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-c88t6_cc8a086f-7f07-4325-a25f-99b769b8ca0f/kube-rbac-proxy-self/0.log"
Sep 16 17:16:17.330371 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.330342    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-c88t6_cc8a086f-7f07-4325-a25f-99b769b8ca0f/openshift-state-metrics/0.log"
Sep 16 17:16:17.375593 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.375563    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/prometheus/0.log"
Sep 16 17:16:17.397544 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.397517    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/config-reloader/0.log"
Sep 16 17:16:17.444659 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.444629    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/thanos-sidecar/0.log"
Sep 16 17:16:17.516250 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.516136    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/kube-rbac-proxy-web/0.log"
Sep 16 17:16:17.572611 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.572578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/kube-rbac-proxy/0.log"
Sep 16 17:16:17.607523 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.607476    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/kube-rbac-proxy-thanos/0.log"
Sep 16 17:16:17.630819 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.630790    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_919737a7-e673-4625-91f6-e2afc6a68852/init-config-reloader/0.log"
Sep 16 17:16:17.722218 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:17.722184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-ntlsd_11ff97db-a382-45ca-af5a-ff918add216a/prometheus-operator-admission-webhook/0.log"
Sep 16 17:16:19.250238 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.250210    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-smsn2_86791c8b-0f46-4f45-a713-7f8dfe9d78fa/networking-console-plugin/0.log"
Sep 16 17:16:19.927468 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.927430    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"]
Sep 16 17:16:19.927958 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.927939    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" containerName="maas-api"
Sep 16 17:16:19.928004 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.927961    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" containerName="maas-api"
Sep 16 17:16:19.928039 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.928028    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42511b19-a03b-4459-9c9d-6a7d7dc2928b" containerName="maas-api"
Sep 16 17:16:19.931537 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.931516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:19.933770 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.933750    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqgpr\"/\"kube-root-ca.crt\""
Sep 16 17:16:19.933770 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.933768    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqgpr\"/\"openshift-service-ca.crt\""
Sep 16 17:16:19.933951 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.933769    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mqgpr\"/\"default-dockercfg-bb76x\""
Sep 16 17:16:19.942598 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:19.942576    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"]
Sep 16 17:16:20.027189 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.027154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgwt\" (UniqueName: \"kubernetes.io/projected/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-kube-api-access-mcgwt\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.027405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.027216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-podres\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.027405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.027238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-proc\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.027405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.027286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-lib-modules\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.027405 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.027338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-sys\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128169 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgwt\" (UniqueName: \"kubernetes.io/projected/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-kube-api-access-mcgwt\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128372 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-podres\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128372 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-proc\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128372 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-lib-modules\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128372 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-sys\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128372 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-proc\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128717 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-lib-modules\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128717 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128416    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-sys\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.128717 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.128378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-podres\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.136152 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.136122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgwt\" (UniqueName: \"kubernetes.io/projected/fd8ee9f6-c2be-4f67-96e9-5b43301bca33-kube-api-access-mcgwt\") pod \"perf-node-gather-daemonset-9klgk\" (UID: \"fd8ee9f6-c2be-4f67-96e9-5b43301bca33\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.243284 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.243188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:20.518182 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:20.518157    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"]
Sep 16 17:16:20.520578 ip-10-0-132-17 kubenswrapper[2577]: W0916 17:16:20.520544    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfd8ee9f6_c2be_4f67_96e9_5b43301bca33.slice/crio-2eb8d64bdd355926a8cdaf4adf63a5c323e7d569eff34f9aaaf56e3f34e3fe40 WatchSource:0}: Error finding container 2eb8d64bdd355926a8cdaf4adf63a5c323e7d569eff34f9aaaf56e3f34e3fe40: Status 404 returned error can't find the container with id 2eb8d64bdd355926a8cdaf4adf63a5c323e7d569eff34f9aaaf56e3f34e3fe40
Sep 16 17:16:21.139770 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:21.139735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk" event={"ID":"fd8ee9f6-c2be-4f67-96e9-5b43301bca33","Type":"ContainerStarted","Data":"a1c678470a4f80c8f6648befbc1edfc5277e23ddb1fd41f3096c72aebefa3865"}
Sep 16 17:16:21.140094 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:21.139795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk" event={"ID":"fd8ee9f6-c2be-4f67-96e9-5b43301bca33","Type":"ContainerStarted","Data":"2eb8d64bdd355926a8cdaf4adf63a5c323e7d569eff34f9aaaf56e3f34e3fe40"}
Sep 16 17:16:21.140094 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:21.139875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:21.154823 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:21.154774    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk" podStartSLOduration=2.154758643 podStartE2EDuration="2.154758643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:16:21.154030893 +0000 UTC m=+2398.239750966" watchObservedRunningTime="2026-09-16 17:16:21.154758643 +0000 UTC m=+2398.240478715"
Sep 16 17:16:22.380756 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:22.380715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b55tq_87ecf712-f78e-4aca-84be-6a24fae7c992/dns/0.log"
Sep 16 17:16:22.402193 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:22.402167    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b55tq_87ecf712-f78e-4aca-84be-6a24fae7c992/kube-rbac-proxy/0.log"
Sep 16 17:16:22.523326 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:22.523296    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2vkc2_e60f338f-568d-49d7-bec5-3e15b5ff8f53/dns-node-resolver/0.log"
Sep 16 17:16:23.147437 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:23.147404    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pkhf5_a0f746c9-f470-40d9-9f34-f495ada964a9/node-ca/0.log"
Sep 16 17:16:24.087826 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:24.087795    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2_dbb947a5-3b59-40e6-8efe-3ede5e7bd29d/istio-proxy/0.log"
Sep 16 17:16:24.516356 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:24.516322    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/0.log"
Sep 16 17:16:24.521664 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:24.521622    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/1.log"
Sep 16 17:16:25.572278 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:25.572245    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w65qs_c6d1931b-2d51-4f56-97bb-45e75ab928af/serve-healthcheck-canary/0.log"
Sep 16 17:16:26.047363 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:26.047332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-25qmb_8fd0d9db-5946-4d2e-9da3-3d0fe8246520/insights-operator/0.log"
Sep 16 17:16:26.048349 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:26.048322    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-25qmb_8fd0d9db-5946-4d2e-9da3-3d0fe8246520/insights-operator/1.log"
Sep 16 17:16:26.212870 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:26.212843    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp2rq_0b7b12f3-3483-4b5e-a990-efdbb2e82402/kube-rbac-proxy/0.log"
Sep 16 17:16:26.233128 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:26.233100    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp2rq_0b7b12f3-3483-4b5e-a990-efdbb2e82402/exporter/0.log"
Sep 16 17:16:26.255347 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:26.255320    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp2rq_0b7b12f3-3483-4b5e-a990-efdbb2e82402/extractor/0.log"
Sep 16 17:16:27.154491 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:27.154458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-9klgk"
Sep 16 17:16:28.298718 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:28.298683    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4xkbz_f3a1c5c9-5019-44bf-8103-3b285fa9893e/manager/0.log"
Sep 16 17:16:28.620356 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:28.620258    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zs5g5_43095e77-16dc-4cec-a9fd-a684496902ed/manager/0.log"
Sep 16 17:16:28.990063 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:28.989972    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-k699z_5ab49632-219f-48f6-9dc2-ae10a6602707/manager/0.log"
Sep 16 17:16:29.013195 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:29.013165    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-k699z_5ab49632-219f-48f6-9dc2-ae10a6602707/copy-manifests/0.log"
Sep 16 17:16:30.417576 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:30.417545    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2_dbb947a5-3b59-40e6-8efe-3ede5e7bd29d/istio-proxy/0.log"
Sep 16 17:16:30.460431 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:30.460378    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c5d8fb6c6-j6wkb_4158574b-8d54-47e3-a17b-4a8c69d924b2/manager/0.log"
Sep 16 17:16:30.819477 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:30.819423    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/0.log"
Sep 16 17:16:30.824653 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:30.824624    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/1.log"
Sep 16 17:16:32.454357 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:32.454316    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5xgd2_dbb947a5-3b59-40e6-8efe-3ede5e7bd29d/istio-proxy/0.log"
Sep 16 17:16:32.855505 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:32.855463    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/0.log"
Sep 16 17:16:32.861053 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:32.861025    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vwq2p_8e5a7999-9a09-4657-bc86-47fa7733c92f/istio-proxy/1.log"
Sep 16 17:16:35.596204 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:35.596169    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-vqffb_4c6d3b0d-bd58-478a-a68c-cdc004050160/migrator/0.log"
Sep 16 17:16:35.617608 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:35.617573    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-vqffb_4c6d3b0d-bd58-478a-a68c-cdc004050160/graceful-termination/0.log"
Sep 16 17:16:37.066951 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.066915    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/kube-multus-additional-cni-plugins/0.log"
Sep 16 17:16:37.087923 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.087889    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/egress-router-binary-copy/0.log"
Sep 16 17:16:37.109569 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.109532    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/cni-plugins/0.log"
Sep 16 17:16:37.130874 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.130839    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/bond-cni-plugin/0.log"
Sep 16 17:16:37.151306 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.151249    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/routeoverride-cni/0.log"
Sep 16 17:16:37.172567 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.172534    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/whereabouts-cni-bincopy/0.log"
Sep 16 17:16:37.192911 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.192880    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lxlc9_fae52386-5999-4306-a8fd-5281677d700e/whereabouts-cni/0.log"
Sep 16 17:16:37.509540 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.509511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qzkn2_8c86855e-120a-4581-9302-9b30b70d1364/kube-multus/0.log"
Sep 16 17:16:37.625846 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.625818    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z78gp_8a7b0509-a48f-420d-964c-caeebd79c4b8/network-metrics-daemon/0.log"
Sep 16 17:16:37.643542 ip-10-0-132-17 kubenswrapper[2577]: I0916 17:16:37.643509    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z78gp_8a7b0509-a48f-420d-964c-caeebd79c4b8/kube-rbac-proxy/0.log"