Sep 18 18:37:11.944402 ip-10-0-132-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 18:37:11.944415 ip-10-0-132-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 18:37:11.944424 ip-10-0-132-229 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 18:37:11.944642 ip-10-0-132-229 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 18:37:22.167145 ip-10-0-132-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 18:37:22.167167 ip-10-0-132-229 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1952dc34de4f4850a99f8ec3b5a7eeaf --
Sep 18 18:39:16.863301 ip-10-0-132-229 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:39:17.234412 ip-10-0-132-229 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:17.234412 ip-10-0-132-229 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:39:17.234412 ip-10-0-132-229 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:17.234412 ip-10-0-132-229 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:39:17.234412 ip-10-0-132-229 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:17.235982 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.235867    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:39:17.240354 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240338    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240356    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240362    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240365    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240368    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240371    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240375    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240378    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240381    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240384    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240388    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240392    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240395    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.240393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240398    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240401    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240405    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240408    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240411    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240414    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240417    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240420    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240423    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240426    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240428    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240431    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240434    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240437    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240440    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240443    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240446    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240449    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240451    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240454    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.240708 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240457    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240459    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240462    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240464    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240467    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240470    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240472    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240475    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240478    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240480    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240483    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240487    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240491    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240494    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240497    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240500    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240502    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240505    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240508    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.241213 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240510    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240514    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240516    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240518    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240521    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240523    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240526    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240528    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240531    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240534    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240536    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240539    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240541    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240544    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240546    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240549    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240552    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240555    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240558    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240562    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.241674 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240565    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240568    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240570    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240573    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240576    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240579    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240582    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240584    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240587    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240590    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240592    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240595    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240597    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240600    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240603    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240605    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240608    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240611    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240613    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240616    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.242301 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240619    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240621    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240624    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240627    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240629    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240632    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.240634    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241330    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241337    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241340    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241344    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241347    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241350    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241352    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241355    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241358    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241361    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241363    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241366    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241369    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241371    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.242941 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241374    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241377    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241380    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241383    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241388    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241392    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241395    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241398    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241401    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241405    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241408    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241411    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241414    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241417    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241419    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241422    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241424    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241427    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241430    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.243457 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241432    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241436    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241439    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241442    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241445    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241448    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241450    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241453    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241456    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241459    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241462    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241464    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241467    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241470    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241473    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241475    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241478    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241480    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241484    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.243943 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241486    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241489    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241491    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241494    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241497    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241499    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241502    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241504    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241507    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241509    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241511    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241514    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241517    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241519    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241522    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241524    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241527    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241529    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241532    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241535    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.244415 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241537    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241540    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241543    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241545    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241548    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241550    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241553    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241555    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241559    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241561    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241564    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241566    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241570    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241572    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241575    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241577    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241580    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241582    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241585    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241587    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.244920 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241589    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241592    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241596    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241600    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241603    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241606    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.241608    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243680    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243693    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243697    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243700    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243703    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243707    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243709    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243712    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243715    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243717    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243720    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243723    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.245488 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243725    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243728    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243730    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243733    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243735    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243738    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243744    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243747    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243750    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243752    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243755    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243757    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243760    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243763    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243765    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243768    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243771    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243773    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243776    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243778    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.246001 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243781    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243785    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243789    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243793    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243796    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243799    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243802    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243804    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243807    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243809    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243812    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243815    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243817    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243820    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243822    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243825    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243827    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243830    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243839    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.246515 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243842    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243845    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243848    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243850    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243853    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243856    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243858    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243861    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243863    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243866    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243870    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243874    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243877    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243880    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243883    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243899    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243902    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243905    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243909    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.246980 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243912    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243915    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243917    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243920    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243922    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243925    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243928    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243930    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243933    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243935    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243938    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243940    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243943    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243951    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243954    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243958    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243962    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243965    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243968    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.247478 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243971    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243974    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243976    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243979    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243981    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243984    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243987    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243990    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243993    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.243996    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244065    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244101    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244110    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244117    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244121    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244124    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244130    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244137    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244141    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244146    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244151    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244155    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:39:17.247942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244159    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244162    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244164    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244167    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244170    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244179    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244185    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244188    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244191    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244194    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244197    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244201    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244204    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244207    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244210    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244213    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244216    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244219    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244222    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244225    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244229    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244232    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244235    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244238    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244240    2558 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:39:17.248468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244243    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244250    2558 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244253    2558 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244256    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244259    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244262    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244265    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244268    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244271    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244273    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244276    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244279    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244282    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244294    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244297    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244300    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244303    2558 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244307    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244310    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244313    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244316    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244319    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244322    2558 flags.go:64] FLAG: --help="false"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244325    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244328    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:39:17.249128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244330    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244333    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244336    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244340    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244342    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244345    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244348    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244351    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244354    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244357    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244359    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244362    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244365    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244368    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244370    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244373    2558 flags.go:64] FLAG: --lock-file=""
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244376    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244379    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244382    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244387    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244389    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244397    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244400    2558 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:39:17.249736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244403    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244406    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244410    2558 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244412    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244416    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244419    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244424    2558 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244427    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244430    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244433    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244435    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244438    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244441    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244444    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244450    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244452    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244455    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244458    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244461    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244466    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244468    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244471    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244474    2558 flags.go:64] FLAG: --port="10250"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244478    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:39:17.250315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244481    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087c96052a20c95b9"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244484    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244486    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244489    2558 flags.go:64] FLAG: --register-node="true"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244492    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244495    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244498    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244506    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244511    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244515    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244518    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244521    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244524    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244526    2558 flags.go:64] FLAG: --runonce="false"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244529    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244532    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244535    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244538    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244541    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244543    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244546    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244549    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244552    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244555    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244558    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244560    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:39:17.250900 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244563    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244566    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244569    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244574    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244577    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244580    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244586    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244588    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244591    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244594    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244597    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244601    2558 flags.go:64] FLAG: --v="2"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244605    2558 flags.go:64] FLAG: --version="false"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244609    2558 flags.go:64] FLAG: --vmodule=""
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244619    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.244623    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244733    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244737    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244740    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244743    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244758    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244761    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244764    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.251564 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244767    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244770    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244772    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244775    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244777    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244780    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244783    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244786    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244789    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244792    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244794    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244797    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244800    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244803    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244805    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244808    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244810    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244813    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244816    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244818    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.252156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244823    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244825    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244828    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244833    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244842    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244845    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244848    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244850    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244853    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244855    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244858    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244860    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244863    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244866    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244868    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244871    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244873    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244876    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244878    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244881    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.252633 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244883    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244900    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244903    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244905    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244908    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244911    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244913    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244916    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244918    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244921    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244924    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244926    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244931    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244933    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244936    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244941    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244943    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244952    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244955    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244957    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.253147 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244960    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244962    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244965    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244968    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244970    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244973    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244975    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244979    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244983    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244987    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244991    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244994    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.244998    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245002    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245005    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245008    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245011    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245014    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245016    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.253634 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245019    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245021    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245024    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245027    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245029    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245033    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245036    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245038    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245042    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245045    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245047    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245055    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.245058    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.245673    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.252623    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:39:17.254106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.252750    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.253380    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.253387    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253447    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253452    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253455    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253458    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253461    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253463    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253466    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253468    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253471    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253473    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253476    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253479    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253482    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253485    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253489    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253492    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.254502 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253495    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253497    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253500    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253503    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253505    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253509    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253512    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253515    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253518    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253521    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253524    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253526    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253529    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253533    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253536    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253539    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253542    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253544    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253547    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.254976 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253550    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253553    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253556    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253558    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253561    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253564    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253566    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253569    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253572    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253574    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253577    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253579    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253582    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253584    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253587    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253589    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253592    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253594    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253597    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253599    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.255449 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253602    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253604    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253607    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253614    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253616    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253619    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253623    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253625    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253628    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253631    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253633    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253636    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253639    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253642    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253644    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253647    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253650    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253652    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253655    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253658    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.255951 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253660    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253663    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253665    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253668    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253670    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253672    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253675    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253677    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253680    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253682    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253685    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253687    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253690    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253692    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253695    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253697    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253702    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253705    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253709    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.256452 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253713    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253715    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253718    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253721    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253723    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.253728    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253857    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253861    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253864    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253867    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253870    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253873    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253877    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253881    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253884    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:17.256984 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253899    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253902    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253905    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253907    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253910    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253913    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253916    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253919    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253922    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253925    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253928    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253932    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253935    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253938    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253941    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253945    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253947    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253950    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253953    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253955    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:17.257371 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253958    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253961    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253963    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253967    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253969    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253971    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253974    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253977    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253979    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253982    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253984    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253987    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253990    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253992    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253995    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.253997    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254000    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254002    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254005    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254007    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:17.257859 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254010    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254012    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254015    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254017    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254020    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254022    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254025    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254028    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254031    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254033    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254036    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254038    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254041    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254044    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254046    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254049    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254051    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254054    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254056    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:17.258484 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254059    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254061    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254064    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254066    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254069    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254071    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254073    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254076    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254078    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254081    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254083    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254086    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254088    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254091    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254093    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254096    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254098    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254101    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254103    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254106    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:17.258948 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254109    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254112    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254115    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254117    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254120    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254123    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254126    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254129    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254133    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254135    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:17.254138    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.254143    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.254264    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.256191    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:39:17.259475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.257016    2558 server.go:1019] "Starting client certificate rotation"
Sep 18 18:39:17.259825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.257113    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:39:17.259825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.257147    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:39:17.259825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.259224    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:39:17.263433 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.263417    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:39:17.277066 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.277044    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:39:17.278640 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.278623    2558 log.go:25] "Validated CRI v1 image API"
Sep 18 18:39:17.279410 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.279390    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:39:17.281904 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.281862    2558 fs.go:135] Filesystem UUIDs: map[58726c85-2621-436c-9131-dde0ef6eed17:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f80935fb-3727-41c6-a4b7-51afc9d95f13:/dev/nvme0n1p3]
Sep 18 18:39:17.281980 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.281884    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:39:17.285204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.285179    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:39:17.287793 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.287681    2558 manager.go:216] Machine: {Timestamp:2026-09-18 18:39:17.285805369 +0000 UTC m=+0.305601985 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101188 MemoryCapacity:32812142592 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec280e156edffeb4bcb4b64b07958cb1 SystemUUID:ec280e15-6edf-feb4-bcb4-b64b07958cb1 BootID:1952dc34-de4f-4850-a99f-8ec3b5a7eeaf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406069248 Type:vfs Inodes:4005388 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ec:75:99:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ec:75:99:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:d8:c9:1a:06:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812142592 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:39:17.287793 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.287790    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:39:17.287929 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.287920    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:39:17.290851 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.290827    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:39:17.291033 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.290855    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:39:17.292223 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.292210    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:39:17.292279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.292228    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:39:17.292279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.292239    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:39:17.292918 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.292906    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:39:17.293006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.292996    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:39:17.295744 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.295730    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:39:17.296073 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296060    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:39:17.296138 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296103    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:39:17.296138 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296119    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:39:17.296138 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296134    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:39:17.296249 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296148    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:39:17.296249 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296160    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:39:17.296533 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296518    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:39:17.296599 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.296542    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:39:17.298529 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.298513    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:39:17.300576 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.300560    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:39:17.300614 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.300595    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:39:17.301961 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301949    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301967    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301973    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301979    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301988    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.301994    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:39:17.302006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302008    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:39:17.302185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302013    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:39:17.302185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302021    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:39:17.302185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302028    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:39:17.302185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302043    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:39:17.302185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.302048    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:39:17.303353 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.303344    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:39:17.303353 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.303353    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:39:17.306136 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.306114    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:39:17.306209 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.306188    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:39:17.306244 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.306235    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:39:17.307059 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.307048    2558 server.go:1268] "Started kubelet"
Sep 18 18:39:17.307172 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.307126    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:39:17.307258 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.307200    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:39:17.307258 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.307148    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:39:17.309555 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.309531    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:39:17.309793 ip-10-0-132-229 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:39:17.312028 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.311461    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:39:17.312489 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.312457    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:39:17.314911 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.314876    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r2p97"
Sep 18 18:39:17.318178 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.318164    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:39:17.318569 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.318550    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:39:17.318724 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.318689    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319368    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319370    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319394    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319375    2558 factory.go:55] Registering systemd factory
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319450    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319484    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:39:17.319437 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319492    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:39:17.319972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319659    2558 factory.go:153] Registering CRI-O factory
Sep 18 18:39:17.319972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319670    2558 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:39:17.319972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319709    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:39:17.319972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319724    2558 factory.go:103] Registering Raw factory
Sep 18 18:39:17.319972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.319735    2558 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:39:17.320188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.320051    2558 manager.go:349] Starting recovery of all containers
Sep 18 18:39:17.320740 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.320716    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r2p97"
Sep 18 18:39:17.326182 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.326162    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 18:39:17.326270 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.326254    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 18:39:17.327010 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.326007    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-229.ec2.internal.18d67e49c9a3d383  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-229.ec2.internal,UID:ip-10-0-132-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-229.ec2.internal,},FirstTimestamp:2026-09-18 18:39:17.307020163 +0000 UTC m=+0.326816813,LastTimestamp:2026-09-18 18:39:17.307020163 +0000 UTC m=+0.326816813,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-229.ec2.internal,}"
Sep 18 18:39:17.328723 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.328709    2558 manager.go:354] Recovery completed
Sep 18 18:39:17.334798 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.334780    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.337853 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.337836    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.337946 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.337882    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.337946 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.337927    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.338548 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.338536    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:39:17.338548 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.338548    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:39:17.338642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.338561    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:39:17.338642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.338622    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.339395 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339381    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.339469 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339409    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.339469 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339433    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.339864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339852    2558 policy_none.go:49] "None policy: Start"
Sep 18 18:39:17.339937 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339867    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:39:17.339937 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.339876    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:39:17.340878 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.340868    2558 policy_none.go:47] "Start"
Sep 18 18:39:17.383535 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.383515    2558 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.383559    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.383574    2558 server.go:87] "Starting device plugin registration server"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.383873    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.383905    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.383995    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.384094    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.384106    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.384795    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:39:17.388279 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.384840    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.426250 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.426217    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:39:17.427551 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.427537    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:39:17.427608 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.427585    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:39:17.427650 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.427612    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:39:17.427696 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.427649    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:39:17.429809 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.429784    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:17.484999 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.484922    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.485933 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.485916    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.485991 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.485951    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.485991 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.485988    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.486054 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.486012    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.493734 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.493717    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.493795 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.493742    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-229.ec2.internal\": node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.508248 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.508228    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.528632 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.528606    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"]
Sep 18 18:39:17.528709 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.528673    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.529569 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.529551    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.529638 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.529578    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.529638 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.529587    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.530982 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.530968    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.531079 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.531120 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531106    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.531669 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531657    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.531719 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531678    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.531719 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531690    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.531779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531720    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.531779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531739    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.531779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.531750    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.532965 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.532950    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.533056 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.532975    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:17.533614 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.533602    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:17.533693 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.533621    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:17.533693 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.533634    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:17.548436 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.548414    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-229.ec2.internal\" not found" node="ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.552179 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.552164    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-229.ec2.internal\" not found" node="ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.609211 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.609186    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.620865 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.620839    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b315991c662748cf7402dbb21807e910-config\") pod \"kube-apiserver-proxy-ip-10-0-132-229.ec2.internal\" (UID: \"b315991c662748cf7402dbb21807e910\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.620954 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.620868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.620954 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.620886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.709873 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.709837    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.721266 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721238    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b315991c662748cf7402dbb21807e910-config\") pod \"kube-apiserver-proxy-ip-10-0-132-229.ec2.internal\" (UID: \"b315991c662748cf7402dbb21807e910\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.721324 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.721324 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721285    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.721389 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.721423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b315991c662748cf7402dbb21807e910-config\") pod \"kube-apiserver-proxy-ip-10-0-132-229.ec2.internal\" (UID: \"b315991c662748cf7402dbb21807e910\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.721423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.721353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dfd54f8c29d2f0dd692098d732d6fef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal\" (UID: \"4dfd54f8c29d2f0dd692098d732d6fef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.810698 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.810668    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:17.852195 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.852164    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.856590 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:17.856426    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:17.911544 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:17.911507    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:18.012041 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.012015    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:18.112639 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.112558    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:18.141031 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.141001    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:18.188652 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.188627    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:18.213650 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.213626    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-229.ec2.internal\" not found"
Sep 18 18:39:18.218690 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.218673    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:18.218781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.218767    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:18.229012 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.228989    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:39:18.229839 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.229827    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"
Sep 18 18:39:18.239171 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.239135    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:39:18.257284 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.257262    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:39:18.257414 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.257393    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:18.257457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.257430    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:18.257457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.257436    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:18.257457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.257433    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:18.296612 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.296588    2558 apiserver.go:52] "Watching apiserver"
Sep 18 18:39:18.310176 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.310125    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:39:18.312547 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.312521    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-g2gmk","openshift-network-diagnostics/network-check-target-6frvx","openshift-network-operator/iptables-alerter-4bh2r","kube-system/global-pull-secret-syncer-5htxl","kube-system/konnectivity-agent-ksjrt","openshift-dns/node-resolver-mcwsf","openshift-image-registry/node-ca-hphd9","openshift-multus/multus-additional-cni-plugins-bzspn","openshift-multus/multus-bmh79","openshift-ovn-kubernetes/ovnkube-node-5zl87","kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr","openshift-cluster-node-tuning-operator/tuned-xmltp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal"]
Sep 18 18:39:18.315054 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.315037    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.317378 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.317359    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.317378 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.317361    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.317546 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.317493    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.317546 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.317497    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-72zz2\""
Sep 18 18:39:18.317657 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.317593    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:39:18.318968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.318954    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:39:18.319508 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.319491    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.319581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.319527    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.319633 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.319610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wbt22\""
Sep 18 18:39:18.320047 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.320031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.322234 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.322218    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.322376 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.322359    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-frb9p\""
Sep 18 18:39:18.322587 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.322574    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.322659 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.322635    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:39:18.322659 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.322641    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.322754 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.322656    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:18.324901 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.324871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-modprobe-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.324968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.324907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysconfig\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.324968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.324923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-kubernetes\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.324968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.324936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-host\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325093 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.324992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-tuned\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325093 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd4bp\" (UniqueName: \"kubernetes.io/projected/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-kube-api-access-xd4bp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325093 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrjcw\" (UniqueName: \"kubernetes.io/projected/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-kube-api-access-xrjcw\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.325093 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325069    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-conf\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325093 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1698d1d8-4716-4902-961b-7dce979703cd-iptables-alerter-script\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmzf\" (UniqueName: \"kubernetes.io/projected/1698d1d8-4716-4902-961b-7dce979703cd-kube-api-access-wrmzf\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-host\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325143    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-serviceca\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325179    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-systemd\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-run\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-sys\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325247    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1698d1d8-4716-4902-961b-7dce979703cd-host-slash\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325278    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-lib-modules\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325323    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-var-lib-kubelet\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325344    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-tmp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.325714 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325368    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 18:34:17 +0000 UTC" deadline="2028-03-09 15:00:50.066754125 +0000 UTC"
Sep 18 18:39:18.325714 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.325416    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12908h21m31.741343229s"
Sep 18 18:39:18.327338 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.327308    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-trd8p\""
Sep 18 18:39:18.327426 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.327362    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:39:18.327465 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.327425    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:39:18.327947 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.327930    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.329496 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.329479    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:39:18.329821 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.329807    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.330000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.329986    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.330186 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.330173    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w992t\""
Sep 18 18:39:18.330719 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.330704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.332860 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.332842    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:39:18.332993 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.332868    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.332993 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.332982    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:39:18.333099 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.332984    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mb86q\""
Sep 18 18:39:18.333209 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.333194    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.333843 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.333826    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.334000 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.333879    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:18.336613 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.336591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:18.336721 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.336653    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:18.338873 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.338857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.341036 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341019    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:39:18.341412 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341385    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:39:18.341510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341416    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.341510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:39:18.341510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341449    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.341510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341494    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:39:18.341510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341432    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kc2ts\""
Sep 18 18:39:18.341856 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.341842    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.343986 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.343884    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9qw7x\""
Sep 18 18:39:18.344074 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.344054    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:39:18.344146 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.344109    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.344278 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.344257    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:39:18.346036 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.346021    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:39:18.346361 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.346348    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vzw98\""
Sep 18 18:39:18.346905 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.346877    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:39:18.346993 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.346881    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:39:18.352695 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.352675    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9p2h"
Sep 18 18:39:18.362030 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.360630    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9p2h"
Sep 18 18:39:18.420277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.420249    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:39:18.425578 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d96ac10-d402-43a3-9aea-2360b5f9f206-hosts-file\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.425672 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-multus-daemon-config\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.425672 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-script-lib\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.425672 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-socket-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.425672 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425662    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-lib-modules\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.425864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-hostroot\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.425864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1698d1d8-4716-4902-961b-7dce979703cd-host-slash\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.425864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425758    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1698d1d8-4716-4902-961b-7dce979703cd-host-slash\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.425864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.425864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d96ac10-d402-43a3-9aea-2360b5f9f206-tmp-dir\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-lib-modules\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-cni-binary-copy\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-bin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.425966    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvcjq\" (UniqueName: \"kubernetes.io/projected/a761273a-cd31-43aa-8363-190dd028344b-kube-api-access-zvcjq\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-modprobe-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34cbd08-703c-4101-95de-a5d18ba8a293-agent-certs\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-device-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.426102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426097    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-modprobe-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34cbd08-703c-4101-95de-a5d18ba8a293-konnectivity-ca\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-socket-dir-parent\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426217    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-serviceca\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-sys-fs\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-etc-kubernetes\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-netd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426408    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-env-overrides\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysconfig\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426472    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-kubernetes\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysconfig\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-tuned\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-system-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-multus-certs\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-kubernetes\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrjcw\" (UniqueName: \"kubernetes.io/projected/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-kube-api-access-xrjcw\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-kubelet-config\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426626    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cnibin\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426658    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-kubelet\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426666    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-serviceca\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-netns\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.426742 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-config\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-conf\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426836    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmzf\" (UniqueName: \"kubernetes.io/projected/1698d1d8-4716-4902-961b-7dce979703cd-kube-api-access-wrmzf\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-systemd\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426904    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-run\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-conf\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-sys\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-run\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-systemd\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-k8s-cni-cncf-io\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427006    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-kubelet\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.426983    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-sys\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427077    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-node-log\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-tmp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9wzk\" (UniqueName: \"kubernetes.io/projected/9bdb7a9a-d783-47ac-aeb1-92b826817290-kube-api-access-b9wzk\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427154    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-system-cni-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.427451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427208    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwnn\" (UniqueName: \"kubernetes.io/projected/2d96ac10-d402-43a3-9aea-2360b5f9f206-kube-api-access-fnwnn\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-systemd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-ovn\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427295    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-log-socket\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-multus\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-conf-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovn-node-metrics-cert\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-dbus\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-etc-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427525    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-bin\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzhgf\" (UniqueName: \"kubernetes.io/projected/1cf56861-37f9-4f77-b75d-50da4476ce75-kube-api-access-vzhgf\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-var-lib-kubelet\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1698d1d8-4716-4902-961b-7dce979703cd-iptables-alerter-script\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-host\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-var-lib-kubelet\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-netns\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427716    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-host\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-var-lib-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chc6j\" (UniqueName: \"kubernetes.io/projected/2d6eec1a-95f1-43c8-83c5-52e5ee107486-kube-api-access-chc6j\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-systemd-units\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-registration-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-host\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.427971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd4bp\" (UniqueName: \"kubernetes.io/projected/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-kube-api-access-xd4bp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-sysctl-d\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428148    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-host\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.428748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428393    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xgt\" (UniqueName: \"kubernetes.io/projected/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-kube-api-access-s6xgt\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.429403 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428246    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1698d1d8-4716-4902-961b-7dce979703cd-iptables-alerter-script\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.429403 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428416    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-cnibin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.429403 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.428438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-slash\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.431111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.431086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-tmp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.431111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.431106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-etc-tuned\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.435280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.435245    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmzf\" (UniqueName: \"kubernetes.io/projected/1698d1d8-4716-4902-961b-7dce979703cd-kube-api-access-wrmzf\") pod \"iptables-alerter-4bh2r\" (UID: \"1698d1d8-4716-4902-961b-7dce979703cd\") " pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.435383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.435356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrjcw\" (UniqueName: \"kubernetes.io/projected/f4e9ae49-7084-4a19-a3d2-9c5fc731343b-kube-api-access-xrjcw\") pod \"node-ca-hphd9\" (UID: \"f4e9ae49-7084-4a19-a3d2-9c5fc731343b\") " pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.435976 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.435959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd4bp\" (UniqueName: \"kubernetes.io/projected/36527054-d1fd-4f68-8cd5-6ef2a67ade5c-kube-api-access-xd4bp\") pod \"tuned-xmltp\" (UID: \"36527054-d1fd-4f68-8cd5-6ef2a67ade5c\") " pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.469393 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.469363    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfd54f8c29d2f0dd692098d732d6fef.slice/crio-0aae82023fd91d6b43d1abd2730f8b54d74194135ca8305813ac8a90c2bc2bfe WatchSource:0}: Error finding container 0aae82023fd91d6b43d1abd2730f8b54d74194135ca8305813ac8a90c2bc2bfe: Status 404 returned error can't find the container with id 0aae82023fd91d6b43d1abd2730f8b54d74194135ca8305813ac8a90c2bc2bfe
Sep 18 18:39:18.470305 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.470282    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb315991c662748cf7402dbb21807e910.slice/crio-0bf319dffa08ac0addf9429c5533fba4a0215c1f5a9a5707c3d27d59fe056bf6 WatchSource:0}: Error finding container 0bf319dffa08ac0addf9429c5533fba4a0215c1f5a9a5707c3d27d59fe056bf6: Status 404 returned error can't find the container with id 0bf319dffa08ac0addf9429c5533fba4a0215c1f5a9a5707c3d27d59fe056bf6
Sep 18 18:39:18.475197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.475182    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:39:18.529344 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529310    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-env-overrides\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529344 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-system-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-multus-certs\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-kubelet-config\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cnibin\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-multus-certs\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-system-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-kubelet\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cnibin\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-kubelet\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-netns\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-config\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.529568 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529491    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-kubelet-config\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-run-netns\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529623    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-cni-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-k8s-cni-cncf-io\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529665    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-kubelet\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-node-log\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-k8s-cni-cncf-io\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529710    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9wzk\" (UniqueName: \"kubernetes.io/projected/9bdb7a9a-d783-47ac-aeb1-92b826817290-kube-api-access-b9wzk\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-system-cni-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529758    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-node-log\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-system-cni-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529749    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-kubelet\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwnn\" (UniqueName: \"kubernetes.io/projected/2d96ac10-d402-43a3-9aea-2360b5f9f206-kube-api-access-fnwnn\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-systemd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-ovn\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.530295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-env-overrides\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-log-socket\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-systemd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529918    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-multus\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-ovn\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529957    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-conf-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.529974    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovn-node-metrics-cert\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-multus\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-conf-dir\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530041    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-log-socket\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530062    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-dbus\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530073    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-config\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-etc-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244d630b-d03b-4e98-8f83-81a041d5eaee-dbus\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-run-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-etc-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-bin\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzhgf\" (UniqueName: \"kubernetes.io/projected/1cf56861-37f9-4f77-b75d-50da4476ce75-kube-api-access-vzhgf\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530180    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-bin\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-netns\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-var-lib-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530252    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-run-netns\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chc6j\" (UniqueName: \"kubernetes.io/projected/2d6eec1a-95f1-43c8-83c5-52e5ee107486-kube-api-access-chc6j\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-systemd-units\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530286    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-var-lib-openvswitch\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-registration-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.531826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530364    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-systemd-units\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-registration-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xgt\" (UniqueName: \"kubernetes.io/projected/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-kube-api-access-s6xgt\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-cnibin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530529    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-cnibin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-slash\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d96ac10-d402-43a3-9aea-2360b5f9f206-hosts-file\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-multus-daemon-config\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530667    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-slash\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-script-lib\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d96ac10-d402-43a3-9aea-2360b5f9f206-hosts-file\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-socket-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-hostroot\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530921    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d96ac10-d402-43a3-9aea-2360b5f9f206-tmp-dir\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.532370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-cni-binary-copy\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-bin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-socket-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.530982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvcjq\" (UniqueName: \"kubernetes.io/projected/a761273a-cd31-43aa-8363-190dd028344b-kube-api-access-zvcjq\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34cbd08-703c-4101-95de-a5d18ba8a293-agent-certs\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.531020    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531046    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-device-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531082    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34cbd08-703c-4101-95de-a5d18ba8a293-konnectivity-ca\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531097    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-socket-dir-parent\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-sys-fs\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-etc-kubernetes\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-netd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531164    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovnkube-script-lib\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531455    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d6eec1a-95f1-43c8-83c5-52e5ee107486-host-cni-netd\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531462    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d96ac10-d402-43a3-9aea-2360b5f9f206-tmp-dir\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-cni-binary-copy\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.533174 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-host-var-lib-cni-bin\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-multus-socket-dir-parent\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.531590    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531596    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-sys-fs\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531639    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-etc-kubernetes\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.531652    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:19.031631446 +0000 UTC m=+2.051428055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.531682    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:19.031661723 +0000 UTC m=+2.051458330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531706    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1cf56861-37f9-4f77-b75d-50da4476ce75-device-dir\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531817    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34cbd08-703c-4101-95de-a5d18ba8a293-konnectivity-ca\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531818    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a761273a-cd31-43aa-8363-190dd028344b-hostroot\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.531936    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.532084    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a761273a-cd31-43aa-8363-190dd028344b-multus-daemon-config\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.532816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d6eec1a-95f1-43c8-83c5-52e5ee107486-ovn-node-metrics-cert\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.534071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.533435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34cbd08-703c-4101-95de-a5d18ba8a293-agent-certs\") pod \"konnectivity-agent-ksjrt\" (UID: \"a34cbd08-703c-4101-95de-a5d18ba8a293\") " pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.538334 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.538305    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:18.538334 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.538331    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:18.538462 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.538341    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:18.538462 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:18.538391    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:19.038378005 +0000 UTC m=+2.058174609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:18.540820 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.540792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzhgf\" (UniqueName: \"kubernetes.io/projected/1cf56861-37f9-4f77-b75d-50da4476ce75-kube-api-access-vzhgf\") pod \"aws-ebs-csi-driver-node-nqqwr\" (UID: \"1cf56861-37f9-4f77-b75d-50da4476ce75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.541058 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.541037    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chc6j\" (UniqueName: \"kubernetes.io/projected/2d6eec1a-95f1-43c8-83c5-52e5ee107486-kube-api-access-chc6j\") pod \"ovnkube-node-5zl87\" (UID: \"2d6eec1a-95f1-43c8-83c5-52e5ee107486\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.541155 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.541139    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvcjq\" (UniqueName: \"kubernetes.io/projected/a761273a-cd31-43aa-8363-190dd028344b-kube-api-access-zvcjq\") pod \"multus-bmh79\" (UID: \"a761273a-cd31-43aa-8363-190dd028344b\") " pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.541213 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.541203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xgt\" (UniqueName: \"kubernetes.io/projected/84549bdb-7ef3-4c2c-8b59-1e76c9114f28-kube-api-access-s6xgt\") pod \"multus-additional-cni-plugins-bzspn\" (UID: \"84549bdb-7ef3-4c2c-8b59-1e76c9114f28\") " pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.541441 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.541422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9wzk\" (UniqueName: \"kubernetes.io/projected/9bdb7a9a-d783-47ac-aeb1-92b826817290-kube-api-access-b9wzk\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:18.541692 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.541675    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwnn\" (UniqueName: \"kubernetes.io/projected/2d96ac10-d402-43a3-9aea-2360b5f9f206-kube-api-access-fnwnn\") pod \"node-resolver-mcwsf\" (UID: \"2d96ac10-d402-43a3-9aea-2360b5f9f206\") " pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.648912 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.648823    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hphd9"
Sep 18 18:39:18.658767 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.658739    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e9ae49_7084_4a19_a3d2_9c5fc731343b.slice/crio-b69e733196a6a84717e0d5adc1fa05a7d293bbff0d297a17d574f64d0aa565ed WatchSource:0}: Error finding container b69e733196a6a84717e0d5adc1fa05a7d293bbff0d297a17d574f64d0aa565ed: Status 404 returned error can't find the container with id b69e733196a6a84717e0d5adc1fa05a7d293bbff0d297a17d574f64d0aa565ed
Sep 18 18:39:18.666342 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.666323    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xmltp"
Sep 18 18:39:18.672281 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.672211    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4bh2r"
Sep 18 18:39:18.674975 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.674865    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36527054_d1fd_4f68_8cd5_6ef2a67ade5c.slice/crio-19203e87a63f816f16d02b31e5a497f6d73da2e4719237b23ecd59ef0c0932c6 WatchSource:0}: Error finding container 19203e87a63f816f16d02b31e5a497f6d73da2e4719237b23ecd59ef0c0932c6: Status 404 returned error can't find the container with id 19203e87a63f816f16d02b31e5a497f6d73da2e4719237b23ecd59ef0c0932c6
Sep 18 18:39:18.681706 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.681657    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1698d1d8_4716_4902_961b_7dce979703cd.slice/crio-dca3469482cb421e3aff22078df903d6e96ca3ddcfe8216d9bc46060bb33f696 WatchSource:0}: Error finding container dca3469482cb421e3aff22078df903d6e96ca3ddcfe8216d9bc46060bb33f696: Status 404 returned error can't find the container with id dca3469482cb421e3aff22078df903d6e96ca3ddcfe8216d9bc46060bb33f696
Sep 18 18:39:18.692425 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.692404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:18.703121 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.703090    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34cbd08_703c_4101_95de_a5d18ba8a293.slice/crio-13209d68ce0ea4fa9a54bcefe89b3954ea307b9720a6ca2b9f3fae22b961ccf2 WatchSource:0}: Error finding container 13209d68ce0ea4fa9a54bcefe89b3954ea307b9720a6ca2b9f3fae22b961ccf2: Status 404 returned error can't find the container with id 13209d68ce0ea4fa9a54bcefe89b3954ea307b9720a6ca2b9f3fae22b961ccf2
Sep 18 18:39:18.707846 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.707828    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mcwsf"
Sep 18 18:39:18.716362 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.716340    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d96ac10_d402_43a3_9aea_2360b5f9f206.slice/crio-ccfe1c20726a54041d169012136f3d0a276661e31f392de1106eea714e302716 WatchSource:0}: Error finding container ccfe1c20726a54041d169012136f3d0a276661e31f392de1106eea714e302716: Status 404 returned error can't find the container with id ccfe1c20726a54041d169012136f3d0a276661e31f392de1106eea714e302716
Sep 18 18:39:18.721823 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.721804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bmh79"
Sep 18 18:39:18.728312 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.728291    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:18.730472 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.730448    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda761273a_cd31_43aa_8363_190dd028344b.slice/crio-f66002977d8912be930dd402512c38a53e401dd00f6c507740f805e7f77e9015 WatchSource:0}: Error finding container f66002977d8912be930dd402512c38a53e401dd00f6c507740f805e7f77e9015: Status 404 returned error can't find the container with id f66002977d8912be930dd402512c38a53e401dd00f6c507740f805e7f77e9015
Sep 18 18:39:18.737180 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.737160    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6eec1a_95f1_43c8_83c5_52e5ee107486.slice/crio-0d7a065433deb06cb3e2d7072f86dbfb48a9decbed5ce5f38ab799505a02a321 WatchSource:0}: Error finding container 0d7a065433deb06cb3e2d7072f86dbfb48a9decbed5ce5f38ab799505a02a321: Status 404 returned error can't find the container with id 0d7a065433deb06cb3e2d7072f86dbfb48a9decbed5ce5f38ab799505a02a321
Sep 18 18:39:18.752411 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.752385    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzspn"
Sep 18 18:39:18.754623 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:18.754605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr"
Sep 18 18:39:18.764269 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.764243    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84549bdb_7ef3_4c2c_8b59_1e76c9114f28.slice/crio-952e82b3fb9f9a946ae9014673034783b39617abc8ce71be47ed64d2d2e32e0b WatchSource:0}: Error finding container 952e82b3fb9f9a946ae9014673034783b39617abc8ce71be47ed64d2d2e32e0b: Status 404 returned error can't find the container with id 952e82b3fb9f9a946ae9014673034783b39617abc8ce71be47ed64d2d2e32e0b
Sep 18 18:39:18.765305 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:18.765282    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf56861_37f9_4f77_b75d_50da4476ce75.slice/crio-ef5fcc68a18819a20820f8d523782a7c14381ba559bed7564513ccbaccf763dd WatchSource:0}: Error finding container ef5fcc68a18819a20820f8d523782a7c14381ba559bed7564513ccbaccf763dd: Status 404 returned error can't find the container with id ef5fcc68a18819a20820f8d523782a7c14381ba559bed7564513ccbaccf763dd
Sep 18 18:39:19.035016 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.034985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:19.035199 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.035030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:19.035199 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.035138    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:19.035311 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.035211    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:20.035194563 +0000 UTC m=+3.054991173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:19.035311 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.035139    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:19.035311 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.035281    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:20.035264181 +0000 UTC m=+3.055060791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:19.135747 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.135711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:19.135941 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.135923    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:19.136050 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.135953    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:19.136050 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.135967    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:19.136050 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.136034    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:20.136013003 +0000 UTC m=+3.155809631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:19.364429 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.364324    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:34:18 +0000 UTC" deadline="2028-06-19 09:20:11.384392416 +0000 UTC"
Sep 18 18:39:19.364429 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.364369    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15350h40m52.020028559s"
Sep 18 18:39:19.429137 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.429099    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:19.429334 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.429252    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:19.430545 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.429674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:19.430545 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:19.429794    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:19.476998 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.467852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal" event={"ID":"b315991c662748cf7402dbb21807e910","Type":"ContainerStarted","Data":"0bf319dffa08ac0addf9429c5533fba4a0215c1f5a9a5707c3d27d59fe056bf6"}
Sep 18 18:39:19.490155 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.485475    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal" event={"ID":"4dfd54f8c29d2f0dd692098d732d6fef","Type":"ContainerStarted","Data":"0aae82023fd91d6b43d1abd2730f8b54d74194135ca8305813ac8a90c2bc2bfe"}
Sep 18 18:39:19.506720 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.506564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerStarted","Data":"952e82b3fb9f9a946ae9014673034783b39617abc8ce71be47ed64d2d2e32e0b"}
Sep 18 18:39:19.537337 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.537190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"0d7a065433deb06cb3e2d7072f86dbfb48a9decbed5ce5f38ab799505a02a321"}
Sep 18 18:39:19.542109 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.542061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bmh79" event={"ID":"a761273a-cd31-43aa-8363-190dd028344b","Type":"ContainerStarted","Data":"f66002977d8912be930dd402512c38a53e401dd00f6c507740f805e7f77e9015"}
Sep 18 18:39:19.573405 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.573349    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mcwsf" event={"ID":"2d96ac10-d402-43a3-9aea-2360b5f9f206","Type":"ContainerStarted","Data":"ccfe1c20726a54041d169012136f3d0a276661e31f392de1106eea714e302716"}
Sep 18 18:39:19.584088 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.583947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4bh2r" event={"ID":"1698d1d8-4716-4902-961b-7dce979703cd","Type":"ContainerStarted","Data":"dca3469482cb421e3aff22078df903d6e96ca3ddcfe8216d9bc46060bb33f696"}
Sep 18 18:39:19.599944 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.599886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hphd9" event={"ID":"f4e9ae49-7084-4a19-a3d2-9c5fc731343b","Type":"ContainerStarted","Data":"b69e733196a6a84717e0d5adc1fa05a7d293bbff0d297a17d574f64d0aa565ed"}
Sep 18 18:39:19.617041 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.616872    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr" event={"ID":"1cf56861-37f9-4f77-b75d-50da4476ce75","Type":"ContainerStarted","Data":"ef5fcc68a18819a20820f8d523782a7c14381ba559bed7564513ccbaccf763dd"}
Sep 18 18:39:19.624910 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.624872    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksjrt" event={"ID":"a34cbd08-703c-4101-95de-a5d18ba8a293","Type":"ContainerStarted","Data":"13209d68ce0ea4fa9a54bcefe89b3954ea307b9720a6ca2b9f3fae22b961ccf2"}
Sep 18 18:39:19.644709 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.644666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xmltp" event={"ID":"36527054-d1fd-4f68-8cd5-6ef2a67ade5c","Type":"ContainerStarted","Data":"19203e87a63f816f16d02b31e5a497f6d73da2e4719237b23ecd59ef0c0932c6"}
Sep 18 18:39:19.850108 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:19.850071    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.043251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.043307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.043435    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.043502    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:22.043482299 +0000 UTC m=+5.063278923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.043910    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:20.046995 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.043959    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:22.043943619 +0000 UTC m=+5.063740241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:20.144611 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.143900    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:20.144611 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.144131    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:20.144611 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.144152    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:20.144611 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.144165    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:20.144611 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.144227    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:22.144206004 +0000 UTC m=+5.164002614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:20.365188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.365090    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:34:18 +0000 UTC" deadline="2028-03-29 07:15:38.057883645 +0000 UTC"
Sep 18 18:39:20.365188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.365133    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13380h36m17.692754691s"
Sep 18 18:39:20.428738 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.428704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:20.428928 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:20.428846    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:20.658347 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:20.658268    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:21.014105 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:21.014019    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:21.316334 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:21.316116    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:21.442823 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:21.442790    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:21.443613 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:21.443027    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:21.443613 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:21.443105    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:21.443732 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:21.443629    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:22.060134    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:22.060193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.060281    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.060348    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.060359    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:26.06033849 +0000 UTC m=+9.080135109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:22.060776 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.060402    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:26.060385795 +0000 UTC m=+9.080182410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:22.160609 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:22.160530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:22.160800 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.160753    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:22.160800 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.160776    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:22.160800 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.160790    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:22.160980 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.160847    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:26.160829262 +0000 UTC m=+9.180625865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:22.429771 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:22.429075    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:22.429771 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:22.429212    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:23.429458 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:23.429383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:23.430106 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:23.429529    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:23.430106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:23.429876    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:23.430106 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:23.430005    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:24.428848 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:24.428357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:24.428848 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:24.428479    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:25.431295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:25.428983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:25.431295 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:25.429122    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:25.431295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:25.429482    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:25.431295 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:25.429573    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:26.093155 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:26.093107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:26.093341 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:26.093171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:26.093341 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.093327    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:26.093460 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.093390    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:34.093371237 +0000 UTC m=+17.113167847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:26.093770 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.093754    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:26.093826 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.093794    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:34.093783795 +0000 UTC m=+17.113580398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:26.194278 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:26.194239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:26.194471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.194388    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:26.194471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.194413    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:26.194471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.194427    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:26.194623 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.194494    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:34.194475639 +0000 UTC m=+17.214272266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:26.428288 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:26.428253    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:26.428474 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:26.428358    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:27.429223 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:27.429185    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:27.429766 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:27.429305    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:27.429766 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:27.429757    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:27.429909 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:27.429862    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:28.429115 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:28.428618    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:28.429115 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:28.428761    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:29.428191 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:29.428146    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:29.428367 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:29.428273    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:29.428367 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:29.428304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:29.428521 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:29.428370    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:30.428442 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:30.428402    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:30.428967 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:30.428536    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:31.428521 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:31.428483    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:31.429004 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:31.428618    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:31.429004 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:31.428688    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:31.429004 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:31.428826    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:32.428475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:32.428438    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:32.428671 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:32.428568    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:33.428638 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:33.428604    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:33.428638 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:33.428625    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:33.429169 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:33.428732    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:33.429169 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:33.428852    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:34.156661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:34.156605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:34.156863 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:34.156704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:34.156863 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.156812    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:34.156998 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.156865    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:50.156850502 +0000 UTC m=+33.176647119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:34.157177 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.157151    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:34.157287 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.157218    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:39:50.157202556 +0000 UTC m=+33.176999170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:34.257999 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:34.257955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:34.258171 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.258116    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:34.258171 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.258136    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:34.258171 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.258144    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:34.258318 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.258199    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:39:50.25818355 +0000 UTC m=+33.277980156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:34.428621 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:34.428539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:34.428762 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:34.428647    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:35.428450 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:35.428417    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:35.428450 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:35.428437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:35.429020 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:35.428543    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:35.429020 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:35.428649    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:36.428376 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:36.428341    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:36.428574 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:36.428458    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:37.428878 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:37.428842    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:37.429336 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:37.428969    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:37.429336 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:37.429061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:37.429336 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:37.429170    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:38.428327 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:38.428295    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:38.428458 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:38.428396    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:39.429189 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.428667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:39.429852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.428690    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:39.429852 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:39.429291    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:39.429852 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:39.429366    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:39.709835 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.709756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xmltp" event={"ID":"36527054-d1fd-4f68-8cd5-6ef2a67ade5c","Type":"ContainerStarted","Data":"ec7e6ed55d7510fc86bb57e0e3105e978ee597640949ec4de7cb6f2ded245a21"}
Sep 18 18:39:39.714083 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.714053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal" event={"ID":"b315991c662748cf7402dbb21807e910","Type":"ContainerStarted","Data":"e0f6cf9c924148c7bf131c49cb3d0882c11be0008ed518f5943535cd207e8810"}
Sep 18 18:39:39.715520 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.715497    2558 generic.go:356] "Generic (PLEG): container finished" podID="4dfd54f8c29d2f0dd692098d732d6fef" containerID="49187ee8a3d931944d24a7eb5defc3272826be64829488722eaa873f7ac9eeae" exitCode=0
Sep 18 18:39:39.715628 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.715561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal" event={"ID":"4dfd54f8c29d2f0dd692098d732d6fef","Type":"ContainerDied","Data":"49187ee8a3d931944d24a7eb5defc3272826be64829488722eaa873f7ac9eeae"}
Sep 18 18:39:39.717026 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.716999    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="66d1e28b8631109daf81d8b22050434f90d9c3399d67a0000eb99069d2c34ccd" exitCode=0
Sep 18 18:39:39.717117 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.717069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"66d1e28b8631109daf81d8b22050434f90d9c3399d67a0000eb99069d2c34ccd"}
Sep 18 18:39:39.720190 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720171    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"b7726ed0fbcb79ee5cd9d7c12588448bf472c4de3d2d08dca93c62020aa080cd"}
Sep 18 18:39:39.720269 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720231    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"5b93178ab106a297651b8f273c98aa4ae81196df6a62b22bb1faddff8f833591"}
Sep 18 18:39:39.720327 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"ead71903a56afe3b14486406d3e972e2cb7f6cb1098451265b4e02c145b0803e"}
Sep 18 18:39:39.720379 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720344    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"c787b69fdd0b81edbec7de294172e7dbb6954446f40267c968f3084dd1b064a5"}
Sep 18 18:39:39.720379 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"1fc876230cad3e7e31a3b969a5386d5e1264a521cd3e0829fe35d1a7f83c67e9"}
Sep 18 18:39:39.720474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.720404    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"d786fd5cd54320b6db4f9c62ada53e566fb92dcc41cafe4d60b7727af6d8bad5"}
Sep 18 18:39:39.721425 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.721404    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bmh79" event={"ID":"a761273a-cd31-43aa-8363-190dd028344b","Type":"ContainerStarted","Data":"45275d34fa2339d181a785c3c9128db507b7dd4f3ce15ca094ee692ce5d4bd78"}
Sep 18 18:39:39.722939 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.722919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mcwsf" event={"ID":"2d96ac10-d402-43a3-9aea-2360b5f9f206","Type":"ContainerStarted","Data":"79f0c1597653d876eb60d941e37835c41941bc9f13980c215ba6b341e4951bbd"}
Sep 18 18:39:39.724293 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.724273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hphd9" event={"ID":"f4e9ae49-7084-4a19-a3d2-9c5fc731343b","Type":"ContainerStarted","Data":"964da18ff8057f63ef4659b872d84d1da88b5c49ef6a5be5cf886d6d08a46996"}
Sep 18 18:39:39.725579 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.725561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr" event={"ID":"1cf56861-37f9-4f77-b75d-50da4476ce75","Type":"ContainerStarted","Data":"c7d530dba88cdec6096ac7e9fe3cfded418fd401f3264e3a4b6588f7f01b5192"}
Sep 18 18:39:39.726863 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.726843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksjrt" event={"ID":"a34cbd08-703c-4101-95de-a5d18ba8a293","Type":"ContainerStarted","Data":"20bad2aa987662f431b89605dbb74c4cc0f9d1f28fb9a67ebd253d53de7090ee"}
Sep 18 18:39:39.727703 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.727668    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xmltp" podStartSLOduration=2.862394938 podStartE2EDuration="22.727656419s" totalImagesPullingTime="19.865261481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.727245258 +0000 UTC m=+22.747041884" watchObservedRunningTime="2026-09-18 18:39:39.727656419 +0000 UTC m=+22.747453045"
Sep 18 18:39:39.743323 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.743275    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bmh79" podStartSLOduration=2.76247791 podStartE2EDuration="22.743259146s" totalImagesPullingTime="19.980781236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.742931919 +0000 UTC m=+22.762728545" watchObservedRunningTime="2026-09-18 18:39:39.743259146 +0000 UTC m=+22.763055773"
Sep 18 18:39:39.773952 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.773869    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-229.ec2.internal" podStartSLOduration=21.773850563 podStartE2EDuration="21.773850563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.773505736 +0000 UTC m=+22.793302364" watchObservedRunningTime="2026-09-18 18:39:39.773850563 +0000 UTC m=+22.793647193"
Sep 18 18:39:39.807736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.807685    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mcwsf" podStartSLOduration=3.012783917 podStartE2EDuration="22.807669324s" totalImagesPullingTime="19.794885407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.807160543 +0000 UTC m=+22.826957169" watchObservedRunningTime="2026-09-18 18:39:39.807669324 +0000 UTC m=+22.827465949"
Sep 18 18:39:39.825860 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.825808    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hphd9" podStartSLOduration=2.9731736079999997 podStartE2EDuration="22.825787821s" totalImagesPullingTime="19.852614213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.825203863 +0000 UTC m=+22.845000488" watchObservedRunningTime="2026-09-18 18:39:39.825787821 +0000 UTC m=+22.845584449"
Sep 18 18:39:39.839441 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.839382    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ksjrt" podStartSLOduration=10.766279181 podStartE2EDuration="22.839360852s" totalImagesPullingTime="12.073081671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:39.839255072 +0000 UTC m=+22.859051696" watchObservedRunningTime="2026-09-18 18:39:39.839360852 +0000 UTC m=+22.859157480"
Sep 18 18:39:39.877144 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.877112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:39.942520 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:39.941561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:40.325480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.325434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:40.362589 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.362564    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:39:40.397388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.397293    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:39:40.362585391Z","UUID":"70f224b9-5b5d-4dc1-8cea-fba29406651d","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:39:40.398648 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.398631    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:39:40.398755 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.398662    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:39:40.426158 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.426130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:40.428383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.428366    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:40.428477 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:40.428453    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:40.730839 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.730806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal" event={"ID":"4dfd54f8c29d2f0dd692098d732d6fef","Type":"ContainerStarted","Data":"694432de0ca5396b52a039e077f44e96d2a58f1749dd21189d16adeb5fd543f6"}
Sep 18 18:39:40.732885 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.732852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4bh2r" event={"ID":"1698d1d8-4716-4902-961b-7dce979703cd","Type":"ContainerStarted","Data":"29a25792422f7cdff66e2d7b04f5c5ad263ae14e6a3654162109db4a358330e0"}
Sep 18 18:39:40.739656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.739619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr" event={"ID":"1cf56861-37f9-4f77-b75d-50da4476ce75","Type":"ContainerStarted","Data":"0cf4f7d79ca4d22e06e873e7e25e84fcd667c3e018cf5aa978b77b968e58026b"}
Sep 18 18:39:40.746790 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.746737    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-229.ec2.internal" podStartSLOduration=22.746720661 podStartE2EDuration="22.746720661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:40.74609451 +0000 UTC m=+23.765891137" watchObservedRunningTime="2026-09-18 18:39:40.746720661 +0000 UTC m=+23.766517282"
Sep 18 18:39:40.759840 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.759792    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4bh2r" podStartSLOduration=3.903646619 podStartE2EDuration="23.759774495s" totalImagesPullingTime="19.856127876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:40.759668793 +0000 UTC m=+23.779465417" watchObservedRunningTime="2026-09-18 18:39:40.759774495 +0000 UTC m=+23.779571120"
Sep 18 18:39:40.879781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.879559    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:40.979389 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:40.979357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:41.427945 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:41.427914    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:41.428139 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:41.428045    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:41.428139 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:41.428095    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:41.428266 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:41.428216    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:41.745022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:41.744926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"f1eed421c786bac04b0665f2dccddcc22e03a304e51bc033223d3a03b6526136"}
Sep 18 18:39:42.428079 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:42.428040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:42.428307 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:42.428180    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:42.749390 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:42.749303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr" event={"ID":"1cf56861-37f9-4f77-b75d-50da4476ce75","Type":"ContainerStarted","Data":"46760dc7ea3149543b948fe5595652b254d0d4268766b61f688f24ebdcc34cb1"}
Sep 18 18:39:42.767509 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:42.767463    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqqwr" podStartSLOduration=3.456075383 podStartE2EDuration="25.767450167s" totalImagesPullingTime="22.311374784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:42.766918507 +0000 UTC m=+25.786715134" watchObservedRunningTime="2026-09-18 18:39:42.767450167 +0000 UTC m=+25.787246793"
Sep 18 18:39:42.855693 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:42.855659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:43.223548 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.223513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:43.311090 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.311053    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:43.376817 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.376782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:43.428377 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.428335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:43.428560 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.428335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:43.428560 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:43.428487    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:43.428673 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:43.428557    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:43.478416 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.478336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:43.943022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.942988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:43.946765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:43.944125    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:44.140661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.140626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:44.205971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.205881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:44.427978 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.427941    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:44.428152 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:44.428071    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:44.557847 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.557816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:44.641092 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.641041    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:44.760131 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.760101    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:44.760699 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.760676    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ksjrt"
Sep 18 18:39:44.825579 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:44.825553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:45.429125 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.428869    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:45.429460 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.428953    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:45.429460 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:45.429166    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:45.429460 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:45.429241    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:45.694873 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.694781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:45.758935 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.758868    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" event={"ID":"2d6eec1a-95f1-43c8-83c5-52e5ee107486","Type":"ContainerStarted","Data":"425e81368cfe32fea2df2cd242883de90419c3bcac1f29f3f299b4a1507338bf"}
Sep 18 18:39:45.759156 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.759135    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:45.760714 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.760691    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="4897c1026f486882eba6bed65303958b291e6adb9f8fbac4f4a773a854163c48" exitCode=0
Sep 18 18:39:45.760823 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.760761    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"4897c1026f486882eba6bed65303958b291e6adb9f8fbac4f4a773a854163c48"}
Sep 18 18:39:45.775712 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.775691    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:45.789142 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.789098    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87" podStartSLOduration=8.953134313 podStartE2EDuration="28.78908546s" totalImagesPullingTime="19.835951147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:45.78886506 +0000 UTC m=+28.808661709" watchObservedRunningTime="2026-09-18 18:39:45.78908546 +0000 UTC m=+28.808882085"
Sep 18 18:39:45.792674 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.792654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:45.886016 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.885982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:45.944227 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:45.944192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:46.428133 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.428106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:46.428273 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:46.428208    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:46.763809 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.763656    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:46.764609 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.764019    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:46.769800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.769333    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6frvx"]
Sep 18 18:39:46.769800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.769442    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:46.769800 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:46.769523    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:46.770070 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.769942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5htxl"]
Sep 18 18:39:46.770070 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.770032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:46.770158 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:46.770135    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:46.771083 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.770881    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2gmk"]
Sep 18 18:39:46.771083 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.770987    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:46.771083 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:46.771051    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:46.781751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:46.781728    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:39:47.768955 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:47.768734    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="0a82bf79f2d48fc55620c6cbdc1a0ffb63b918987c6ef4315ce3e3f1219fac53" exitCode=0
Sep 18 18:39:47.769447 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:47.768806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"0a82bf79f2d48fc55620c6cbdc1a0ffb63b918987c6ef4315ce3e3f1219fac53"}
Sep 18 18:39:48.428333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:48.428299    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:48.428495 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:48.428410    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:48.428495 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:48.428417    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:48.428495 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:48.428441    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:48.428601 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:48.428523    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:48.428601 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:48.428590    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:48.773367 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:48.773279    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="1ce34fc93eaec3a0c24a12405c9c3f4372b41afbc7a3b1678a431cdf2998cef0" exitCode=0
Sep 18 18:39:48.773716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:48.773368    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"1ce34fc93eaec3a0c24a12405c9c3f4372b41afbc7a3b1678a431cdf2998cef0"}
Sep 18 18:39:50.176601 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.176567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:50.177082 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.176609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:50.177082 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.176713    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:50.177082 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.176738    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:50.177082 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.176796    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret podName:244d630b-d03b-4e98-8f83-81a041d5eaee nodeName:}" failed. No retries permitted until 2026-09-18 18:40:22.176776275 +0000 UTC m=+65.196572884 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret") pod "global-pull-secret-syncer-5htxl" (UID: "244d630b-d03b-4e98-8f83-81a041d5eaee") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:50.177082 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.176817    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs podName:9bdb7a9a-d783-47ac-aeb1-92b826817290 nodeName:}" failed. No retries permitted until 2026-09-18 18:40:22.176806965 +0000 UTC m=+65.196603575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs") pod "network-metrics-daemon-g2gmk" (UID: "9bdb7a9a-d783-47ac-aeb1-92b826817290") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:50.277483 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.277450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:50.277694 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.277634    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:50.277694 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.277656    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:50.277694 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.277667    2558 projected.go:196] Error preparing data for projected volume kube-api-access-nrkfh for pod openshift-network-diagnostics/network-check-target-6frvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:50.277834 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.277725    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh podName:a59d9eb2-9eb0-42c7-b656-49679e3b164b nodeName:}" failed. No retries permitted until 2026-09-18 18:40:22.277710834 +0000 UTC m=+65.297507437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkfh" (UniqueName: "kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh") pod "network-check-target-6frvx" (UID: "a59d9eb2-9eb0-42c7-b656-49679e3b164b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:50.428870 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.428786    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:50.429034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.428802    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:50.429034 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.428922    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6frvx" podUID="a59d9eb2-9eb0-42c7-b656-49679e3b164b"
Sep 18 18:39:50.429034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:50.428935    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:50.429182 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.429053    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5htxl" podUID="244d630b-d03b-4e98-8f83-81a041d5eaee"
Sep 18 18:39:50.429182 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:50.429130    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2gmk" podUID="9bdb7a9a-d783-47ac-aeb1-92b826817290"
Sep 18 18:39:51.128199 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:51.128172    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mcwsf_2d96ac10-d402-43a3-9aea-2360b5f9f206/dns-node-resolver/0.log"
Sep 18 18:39:52.130259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.130191    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hphd9_f4e9ae49-7084-4a19-a3d2-9c5fc731343b/node-ca/0.log"
Sep 18 18:39:52.305552 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.305517    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-229.ec2.internal" event="NodeReady"
Sep 18 18:39:52.305727 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.305671    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:39:52.343904 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.343851    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:39:52.346210 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.346188    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.348505 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.348484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 18:39:52.348632 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.348614    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 18:39:52.348700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.348484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 18:39:52.348700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.348656    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dvnsn\""
Sep 18 18:39:52.353650 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.353471    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 18:39:52.353756 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.353681    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dxrcs"]
Sep 18 18:39:52.355253 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.355235    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.357221 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.357171    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ljbnj\""
Sep 18 18:39:52.357472 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.357454    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:39:52.357603 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.357578    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:39:52.357703 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.357666    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:39:52.361001 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.360979    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8gwhh"]
Sep 18 18:39:52.366689 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.365317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:39:52.366689 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.365462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.366689 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.365495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dxrcs"]
Sep 18 18:39:52.368041 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.368016    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:39:52.368447 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.368296    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:39:52.368447 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.368324    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-swhrv\""
Sep 18 18:39:52.368447 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.368430    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:39:52.368832 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.368811    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:39:52.374927 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.374885    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8gwhh"]
Sep 18 18:39:52.375515 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.375498    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nxqvl"]
Sep 18 18:39:52.377494 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.377476    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.380938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.380866    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r76nj\""
Sep 18 18:39:52.380938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.380880    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:39:52.380938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.380905    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:39:52.391262 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.391237    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nxqvl"]
Sep 18 18:39:52.392114 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392334 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392334 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392436 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc8h6\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.392436 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.392372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.428107 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.428073    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:39:52.428107 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.428107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:39:52.428357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.428190    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:39:52.430779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.430626    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:39:52.430779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.430673    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:39:52.430779 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.430676    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mb7g\""
Sep 18 18:39:52.431024 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.430834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:39:52.431295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.431258    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:39:52.432203 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.432186    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kbgtj\""
Sep 18 18:39:52.493031 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.492989    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.493031 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.493273 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.493273 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbvj2\" (UniqueName: \"kubernetes.io/projected/fb7d786a-44b9-4939-99eb-07c8655f9b08-kube-api-access-dbvj2\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.493273 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493253    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847fh\" (UniqueName: \"kubernetes.io/projected/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-api-access-847fh\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.493438 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-config-volume\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.493438 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.493438 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.493438 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493443    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493459    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.493465    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.493479    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bb87766b-lbcz6: secret "image-registry-tls" not found
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493503    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48432861-84d0-48ef-a69c-f3e49ca4110d-crio-socket\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.493523    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls podName:962d2637-c781-468b-9bbb-bc4ed05342f8 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:52.993507442 +0000 UTC m=+36.013304046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls") pod "image-registry-8bb87766b-lbcz6" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8") : secret "image-registry-tls" not found
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493555    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48432861-84d0-48ef-a69c-f3e49ca4110d-data-volume\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-tmp-dir\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7d786a-44b9-4939-99eb-07c8655f9b08-cert\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.493696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493629    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbzv\" (UniqueName: \"kubernetes.io/projected/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-kube-api-access-fdbzv\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.494185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.494185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493711    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.494185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.494185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc8h6\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.494185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.493797    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.494776 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.494732    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.498166 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.498141    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.498264 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.498196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.502229 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.502196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.502426 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.502410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc8h6\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.594554 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.594554 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594556    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48432861-84d0-48ef-a69c-f3e49ca4110d-crio-socket\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48432861-84d0-48ef-a69c-f3e49ca4110d-data-volume\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-tmp-dir\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7d786a-44b9-4939-99eb-07c8655f9b08-cert\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.594686    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbzv\" (UniqueName: \"kubernetes.io/projected/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-kube-api-access-fdbzv\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.594705    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.594752    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls podName:48432861-84d0-48ef-a69c-f3e49ca4110d nodeName:}" failed. No retries permitted until 2026-09-18 18:39:53.09473207 +0000 UTC m=+36.114528677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8gwhh" (UID: "48432861-84d0-48ef-a69c-f3e49ca4110d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:52.594781 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.594774    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls podName:5d636b83-5cbf-40f3-9abf-c312f1fbb6ac nodeName:}" failed. No retries permitted until 2026-09-18 18:39:53.094764274 +0000 UTC m=+36.114560884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls") pod "dns-default-nxqvl" (UID: "5d636b83-5cbf-40f3-9abf-c312f1fbb6ac") : secret "dns-default-metrics-tls" not found
Sep 18 18:39:52.595173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.595173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594849    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbvj2\" (UniqueName: \"kubernetes.io/projected/fb7d786a-44b9-4939-99eb-07c8655f9b08-kube-api-access-dbvj2\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.595173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847fh\" (UniqueName: \"kubernetes.io/projected/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-api-access-847fh\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.595173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.594913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-config-volume\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.595173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.595098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48432861-84d0-48ef-a69c-f3e49ca4110d-data-volume\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.595340 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.595283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-tmp-dir\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.595578 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.595557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-config-volume\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.595663 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.595568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.595784 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.595746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48432861-84d0-48ef-a69c-f3e49ca4110d-crio-socket\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.598137 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.598119    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7d786a-44b9-4939-99eb-07c8655f9b08-cert\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.603192 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.603168    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbzv\" (UniqueName: \"kubernetes.io/projected/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-kube-api-access-fdbzv\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:52.604011 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.603983    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847fh\" (UniqueName: \"kubernetes.io/projected/48432861-84d0-48ef-a69c-f3e49ca4110d-kube-api-access-847fh\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:52.604287 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.604265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbvj2\" (UniqueName: \"kubernetes.io/projected/fb7d786a-44b9-4939-99eb-07c8655f9b08-kube-api-access-dbvj2\") pod \"ingress-canary-dxrcs\" (UID: \"fb7d786a-44b9-4939-99eb-07c8655f9b08\") " pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.671517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.671427    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dxrcs"
Sep 18 18:39:52.997952 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:52.997908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:52.998118 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.998095    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:39:52.998118 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.998117    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bb87766b-lbcz6: secret "image-registry-tls" not found
Sep 18 18:39:52.998195 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:52.998186    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls podName:962d2637-c781-468b-9bbb-bc4ed05342f8 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:53.998167576 +0000 UTC m=+37.017964181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls") pod "image-registry-8bb87766b-lbcz6" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8") : secret "image-registry-tls" not found
Sep 18 18:39:53.098683 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:53.098638    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:53.098859 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:53.098693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:53.098859 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:53.098818    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:39:53.098859 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:53.098827    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:53.099023 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:53.098918    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls podName:5d636b83-5cbf-40f3-9abf-c312f1fbb6ac nodeName:}" failed. No retries permitted until 2026-09-18 18:39:54.098879529 +0000 UTC m=+37.118676149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls") pod "dns-default-nxqvl" (UID: "5d636b83-5cbf-40f3-9abf-c312f1fbb6ac") : secret "dns-default-metrics-tls" not found
Sep 18 18:39:53.099023 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:53.098940    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls podName:48432861-84d0-48ef-a69c-f3e49ca4110d nodeName:}" failed. No retries permitted until 2026-09-18 18:39:54.09892922 +0000 UTC m=+37.118725827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8gwhh" (UID: "48432861-84d0-48ef-a69c-f3e49ca4110d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:54.006402 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.006093    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:54.006402 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.006254    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:39:54.006402 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.006392    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bb87766b-lbcz6: secret "image-registry-tls" not found
Sep 18 18:39:54.007364 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.006461    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls podName:962d2637-c781-468b-9bbb-bc4ed05342f8 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:56.006441626 +0000 UTC m=+39.026238230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls") pod "image-registry-8bb87766b-lbcz6" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8") : secret "image-registry-tls" not found
Sep 18 18:39:54.007364 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.007330    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dxrcs"]
Sep 18 18:39:54.010967 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:39:54.010943    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7d786a_44b9_4939_99eb_07c8655f9b08.slice/crio-c758a0329149307c6507e321466f7011a9d770b029e64951e120cb5b8a1ed940 WatchSource:0}: Error finding container c758a0329149307c6507e321466f7011a9d770b029e64951e120cb5b8a1ed940: Status 404 returned error can't find the container with id c758a0329149307c6507e321466f7011a9d770b029e64951e120cb5b8a1ed940
Sep 18 18:39:54.107241 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.107210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:54.107241 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.107243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:54.107433 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.107362    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:39:54.107433 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.107361    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:54.107433 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.107418    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls podName:5d636b83-5cbf-40f3-9abf-c312f1fbb6ac nodeName:}" failed. No retries permitted until 2026-09-18 18:39:56.107403141 +0000 UTC m=+39.127199751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls") pod "dns-default-nxqvl" (UID: "5d636b83-5cbf-40f3-9abf-c312f1fbb6ac") : secret "dns-default-metrics-tls" not found
Sep 18 18:39:54.107433 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:54.107434    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls podName:48432861-84d0-48ef-a69c-f3e49ca4110d nodeName:}" failed. No retries permitted until 2026-09-18 18:39:56.107426268 +0000 UTC m=+39.127222872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8gwhh" (UID: "48432861-84d0-48ef-a69c-f3e49ca4110d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:54.791919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.791787    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dxrcs" event={"ID":"fb7d786a-44b9-4939-99eb-07c8655f9b08","Type":"ContainerStarted","Data":"c758a0329149307c6507e321466f7011a9d770b029e64951e120cb5b8a1ed940"}
Sep 18 18:39:54.795187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.795158    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="f52e0c397954b63f1618f6d9f436faf7c7b2193e3e7643dc679e53af836054d2" exitCode=0
Sep 18 18:39:54.795314 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:54.795224    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"f52e0c397954b63f1618f6d9f436faf7c7b2193e3e7643dc679e53af836054d2"}
Sep 18 18:39:55.802967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:55.802930    2558 generic.go:356] "Generic (PLEG): container finished" podID="84549bdb-7ef3-4c2c-8b59-1e76c9114f28" containerID="5c2dc94f8d0637e56e7c25210370cacb2ea25a32b7fad392f31f52f824fefca2" exitCode=0
Sep 18 18:39:55.803717 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:55.803002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerDied","Data":"5c2dc94f8d0637e56e7c25210370cacb2ea25a32b7fad392f31f52f824fefca2"}
Sep 18 18:39:56.023946 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.023910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:39:56.024104 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.024075    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:39:56.024104 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.024095    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bb87766b-lbcz6: secret "image-registry-tls" not found
Sep 18 18:39:56.024206 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.024165    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls podName:962d2637-c781-468b-9bbb-bc4ed05342f8 nodeName:}" failed. No retries permitted until 2026-09-18 18:40:00.024141823 +0000 UTC m=+43.043938447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls") pod "image-registry-8bb87766b-lbcz6" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8") : secret "image-registry-tls" not found
Sep 18 18:39:56.125255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.125171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:39:56.125255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.125224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:39:56.125471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.125308    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:56.125471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.125373    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls podName:48432861-84d0-48ef-a69c-f3e49ca4110d nodeName:}" failed. No retries permitted until 2026-09-18 18:40:00.12535667 +0000 UTC m=+43.145153274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8gwhh" (UID: "48432861-84d0-48ef-a69c-f3e49ca4110d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:39:56.125471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.125396    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:39:56.125471 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:39:56.125455    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls podName:5d636b83-5cbf-40f3-9abf-c312f1fbb6ac nodeName:}" failed. No retries permitted until 2026-09-18 18:40:00.125440832 +0000 UTC m=+43.145237438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls") pod "dns-default-nxqvl" (UID: "5d636b83-5cbf-40f3-9abf-c312f1fbb6ac") : secret "dns-default-metrics-tls" not found
Sep 18 18:39:56.809631 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.809595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzspn" event={"ID":"84549bdb-7ef3-4c2c-8b59-1e76c9114f28","Type":"ContainerStarted","Data":"fd2179701c07d84a941c815fa77641d017e9d9af661ed1490baac208073d2c5c"}
Sep 18 18:39:56.811197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.811174    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dxrcs" event={"ID":"fb7d786a-44b9-4939-99eb-07c8655f9b08","Type":"ContainerStarted","Data":"7f194215ad2a51c7df2d68f0a90f9140868ae1a6d9494b91b7d97a2ca3f8aef7"}
Sep 18 18:39:56.832958 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.832903    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bzspn" podStartSLOduration=7.896904148 podStartE2EDuration="39.832872101s" totalImagesPullingTime="31.935967953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:56.831914737 +0000 UTC m=+39.851711358" watchObservedRunningTime="2026-09-18 18:39:56.832872101 +0000 UTC m=+39.852668726"
Sep 18 18:39:56.849426 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:39:56.848824    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dxrcs" podStartSLOduration=2.197918259 podStartE2EDuration="4.848800518s" totalImagesPullingTime="2.650882259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:56.848089682 +0000 UTC m=+39.867886299" watchObservedRunningTime="2026-09-18 18:39:56.848800518 +0000 UTC m=+39.868597146"
Sep 18 18:40:00.058760 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.058714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:00.062133 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.062099    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"image-registry-8bb87766b-lbcz6\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") " pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:00.159254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.159213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:40:00.159254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.159253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:40:00.160801 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.160774    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:00.161789 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.161769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d636b83-5cbf-40f3-9abf-c312f1fbb6ac-metrics-tls\") pod \"dns-default-nxqvl\" (UID: \"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac\") " pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:40:00.161874 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.161803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48432861-84d0-48ef-a69c-f3e49ca4110d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8gwhh\" (UID: \"48432861-84d0-48ef-a69c-f3e49ca4110d\") " pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:40:00.181852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.181816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8gwhh"
Sep 18 18:40:00.188973 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.188944    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:40:00.463143 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.463110    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:40:00.466464 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:00.466441    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962d2637_c781_468b_9bbb_bc4ed05342f8.slice/crio-233830082a46ac8a99282cf3b5a77277fa173febbf89abdf4fb6b14584004c4a WatchSource:0}: Error finding container 233830082a46ac8a99282cf3b5a77277fa173febbf89abdf4fb6b14584004c4a: Status 404 returned error can't find the container with id 233830082a46ac8a99282cf3b5a77277fa173febbf89abdf4fb6b14584004c4a
Sep 18 18:40:00.492393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.492362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8gwhh"]
Sep 18 18:40:00.496638 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:00.496610    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48432861_84d0_48ef_a69c_f3e49ca4110d.slice/crio-44d592e462672d8764df6731a4344a0a1b2eccf68d8139d5068ee4e0cb8d425f WatchSource:0}: Error finding container 44d592e462672d8764df6731a4344a0a1b2eccf68d8139d5068ee4e0cb8d425f: Status 404 returned error can't find the container with id 44d592e462672d8764df6731a4344a0a1b2eccf68d8139d5068ee4e0cb8d425f
Sep 18 18:40:00.528707 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.528680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nxqvl"]
Sep 18 18:40:00.531517 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:00.531482    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d636b83_5cbf_40f3_9abf_c312f1fbb6ac.slice/crio-0194efcaaea14d4d282ded3b19a311dcbfcd81dd25cfb8f091f2a5b7ab26ee51 WatchSource:0}: Error finding container 0194efcaaea14d4d282ded3b19a311dcbfcd81dd25cfb8f091f2a5b7ab26ee51: Status 404 returned error can't find the container with id 0194efcaaea14d4d282ded3b19a311dcbfcd81dd25cfb8f091f2a5b7ab26ee51
Sep 18 18:40:00.822640 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.822606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" event={"ID":"962d2637-c781-468b-9bbb-bc4ed05342f8","Type":"ContainerStarted","Data":"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"}
Sep 18 18:40:00.822827 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.822694    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" event={"ID":"962d2637-c781-468b-9bbb-bc4ed05342f8","Type":"ContainerStarted","Data":"233830082a46ac8a99282cf3b5a77277fa173febbf89abdf4fb6b14584004c4a"}
Sep 18 18:40:00.822827 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.822719    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:00.823979 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.823946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxqvl" event={"ID":"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac","Type":"ContainerStarted","Data":"0194efcaaea14d4d282ded3b19a311dcbfcd81dd25cfb8f091f2a5b7ab26ee51"}
Sep 18 18:40:00.825467 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.825443    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8gwhh" event={"ID":"48432861-84d0-48ef-a69c-f3e49ca4110d","Type":"ContainerStarted","Data":"504a0affd451746545d9db454fc7d2ec8944b3acc344eead7b3b655ad494ddc7"}
Sep 18 18:40:00.825561 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.825498    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8gwhh" event={"ID":"48432861-84d0-48ef-a69c-f3e49ca4110d","Type":"ContainerStarted","Data":"44d592e462672d8764df6731a4344a0a1b2eccf68d8139d5068ee4e0cb8d425f"}
Sep 18 18:40:00.864157 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:00.864095    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" podStartSLOduration=17.864071086 podStartE2EDuration="17.864071086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:00.863432322 +0000 UTC m=+43.883228967" watchObservedRunningTime="2026-09-18 18:40:00.864071086 +0000 UTC m=+43.883867713"
Sep 18 18:40:02.832100 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:02.832067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8gwhh" event={"ID":"48432861-84d0-48ef-a69c-f3e49ca4110d","Type":"ContainerStarted","Data":"6fbe2cb56005cb2cd423de5dc449afb959a705103427094da7240653457c50fb"}
Sep 18 18:40:03.843158 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:03.843108    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxqvl" event={"ID":"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac","Type":"ContainerStarted","Data":"f30af8b9ea81619f8c37ef7c638091bed4e50409d29541ad0c492fa2b65d3572"}
Sep 18 18:40:03.843642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:03.843217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxqvl" event={"ID":"5d636b83-5cbf-40f3-9abf-c312f1fbb6ac","Type":"ContainerStarted","Data":"70ddc6e6aedd48152dc45a90a40f2c1dfbcad33dfd8d19cdf7357153100dbfbf"}
Sep 18 18:40:03.843642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:03.843301    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:40:03.866971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:03.866908    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nxqvl" podStartSLOduration=9.56265091 podStartE2EDuration="11.86686963s" totalImagesPullingTime="2.30421872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:03.865228645 +0000 UTC m=+46.885025272" watchObservedRunningTime="2026-09-18 18:40:03.86686963 +0000 UTC m=+46.886666283"
Sep 18 18:40:04.848339 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:04.848294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8gwhh" event={"ID":"48432861-84d0-48ef-a69c-f3e49ca4110d","Type":"ContainerStarted","Data":"10f0435074ca96cd1c77e5f93e4b7a33d306814ac8efe05b28935e989f367feb"}
Sep 18 18:40:04.865969 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:04.865916    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8gwhh" podStartSLOduration=9.14070403 podStartE2EDuration="12.865878493s" totalImagesPullingTime="3.725174463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:04.864686708 +0000 UTC m=+47.884483332" watchObservedRunningTime="2026-09-18 18:40:04.865878493 +0000 UTC m=+47.885675118"
Sep 18 18:40:11.481802 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.481754    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"]
Sep 18 18:40:11.483617 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.483601    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"]
Sep 18 18:40:11.483739 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.483721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.485302 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.485282    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"]
Sep 18 18:40:11.485449 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.485428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.486600 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.486579    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 18:40:11.486938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.486917    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.487345 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487228    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 18 18:40:11.487345 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487303    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 18:40:11.487345 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487305    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 18:40:11.487558 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487306    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mpjp4\""
Sep 18 18:40:11.487558 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487330    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 18:40:11.487842 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 18:40:11.487933 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.487852    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 18:40:11.488031 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.488018    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 18:40:11.488478 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.488460    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:40:11.490122 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.490104    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 18:40:11.502364 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.502339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"]
Sep 18 18:40:11.503052 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.503032    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"]
Sep 18 18:40:11.507061 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.507035    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"]
Sep 18 18:40:11.534586 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.534540    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-687dbbd5d8-hvg4r"]
Sep 18 18:40:11.536446 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.536428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.553468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.553444    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687dbbd5d8-hvg4r"]
Sep 18 18:40:11.646526 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646486    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-image-registry-private-configuration\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.646526 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646523    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkqq7\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-kube-api-access-vkqq7\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrdbm\" (UniqueName: \"kubernetes.io/projected/a9623c6a-3a5b-452b-a8e2-a437692b9a98-kube-api-access-mrdbm\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-trusted-ca\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mth4f\" (UniqueName: \"kubernetes.io/projected/72574134-252d-44ec-bc3b-7bd3bbb22350-kube-api-access-mth4f\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.646751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-certificates\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-installation-pull-secrets\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-bound-sa-token\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646803    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646848    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/047b794b-3638-44f6-a79f-610ca0a5736d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646867    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9623c6a-3a5b-452b-a8e2-a437692b9a98-tmp\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9623c6a-3a5b-452b-a8e2-a437692b9a98-klusterlet-config\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xwr\" (UniqueName: \"kubernetes.io/projected/047b794b-3638-44f6-a79f-610ca0a5736d-kube-api-access-22xwr\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-tls\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646963    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872f73e0-7d28-4967-8473-6276d1c7ebbd-ca-trust-extracted\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.647000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.646994    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.647277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.647011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72574134-252d-44ec-bc3b-7bd3bbb22350-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.748411 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.748497 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mth4f\" (UniqueName: \"kubernetes.io/projected/72574134-252d-44ec-bc3b-7bd3bbb22350-kube-api-access-mth4f\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.748497 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-certificates\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.748497 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-installation-pull-secrets\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.748497 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748494    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-bound-sa-token\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.748666 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.748666 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748553    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/047b794b-3638-44f6-a79f-610ca0a5736d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.748666 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9623c6a-3a5b-452b-a8e2-a437692b9a98-tmp\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.748800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9623c6a-3a5b-452b-a8e2-a437692b9a98-klusterlet-config\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.748849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xwr\" (UniqueName: \"kubernetes.io/projected/047b794b-3638-44f6-a79f-610ca0a5736d-kube-api-access-22xwr\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.748935 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-tls\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.748935 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872f73e0-7d28-4967-8473-6276d1c7ebbd-ca-trust-extracted\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.749038 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.748977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.749104 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72574134-252d-44ec-bc3b-7bd3bbb22350-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.749104 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-image-registry-private-configuration\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.749194 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkqq7\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-kube-api-access-vkqq7\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.749194 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749177    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrdbm\" (UniqueName: \"kubernetes.io/projected/a9623c6a-3a5b-452b-a8e2-a437692b9a98-kube-api-access-mrdbm\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.749315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749233    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-trusted-ca\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.749479 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9623c6a-3a5b-452b-a8e2-a437692b9a98-tmp\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.749690 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.750277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-certificates\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.750277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.749971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872f73e0-7d28-4967-8473-6276d1c7ebbd-ca-trust-extracted\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.750617 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.750276    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/047b794b-3638-44f6-a79f-610ca0a5736d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.753425 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.753370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-registry-tls\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.753925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.753878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-image-registry-private-configuration\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.754455 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.754386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9623c6a-3a5b-452b-a8e2-a437692b9a98-klusterlet-config\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.754548 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.754438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.754548 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.754528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72574134-252d-44ec-bc3b-7bd3bbb22350-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.755086 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.754992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872f73e0-7d28-4967-8473-6276d1c7ebbd-installation-pull-secrets\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.755384 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.755355    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872f73e0-7d28-4967-8473-6276d1c7ebbd-trusted-ca\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.755536 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.755519    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.755625 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.755523    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-ca\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.758840 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.758817    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/047b794b-3638-44f6-a79f-610ca0a5736d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.760246 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.760221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-bound-sa-token\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.760538 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.760477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mth4f\" (UniqueName: \"kubernetes.io/projected/72574134-252d-44ec-bc3b-7bd3bbb22350-kube-api-access-mth4f\") pod \"managed-serviceaccount-addon-agent-64bb6c5996-bqlxj\" (UID: \"72574134-252d-44ec-bc3b-7bd3bbb22350\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.761318 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.761293    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrdbm\" (UniqueName: \"kubernetes.io/projected/a9623c6a-3a5b-452b-a8e2-a437692b9a98-kube-api-access-mrdbm\") pod \"klusterlet-addon-workmgr-64665b9646-ggbzg\" (UID: \"a9623c6a-3a5b-452b-a8e2-a437692b9a98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.761827 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.761795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xwr\" (UniqueName: \"kubernetes.io/projected/047b794b-3638-44f6-a79f-610ca0a5736d-kube-api-access-22xwr\") pod \"cluster-proxy-proxy-agent-6fd9964594-b5wdz\" (UID: \"047b794b-3638-44f6-a79f-610ca0a5736d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.762354 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.762334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkqq7\" (UniqueName: \"kubernetes.io/projected/872f73e0-7d28-4967-8473-6276d1c7ebbd-kube-api-access-vkqq7\") pod \"image-registry-687dbbd5d8-hvg4r\" (UID: \"872f73e0-7d28-4967-8473-6276d1c7ebbd\") " pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:11.802411 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.802377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"
Sep 18 18:40:11.810503 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.810479    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"
Sep 18 18:40:11.817590 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.817566    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:11.855105 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:11.855073    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:12.077827 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.077792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj"]
Sep 18 18:40:12.080514 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:12.080487    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72574134_252d_44ec_bc3b_7bd3bbb22350.slice/crio-766e2e9445eaec3a4597d51b2a8dca230e6f483a79661fad7684eadee4c50582 WatchSource:0}: Error finding container 766e2e9445eaec3a4597d51b2a8dca230e6f483a79661fad7684eadee4c50582: Status 404 returned error can't find the container with id 766e2e9445eaec3a4597d51b2a8dca230e6f483a79661fad7684eadee4c50582
Sep 18 18:40:12.095611 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.095586    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz"]
Sep 18 18:40:12.099002 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:12.098976    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod047b794b_3638_44f6_a79f_610ca0a5736d.slice/crio-dcd687a25da90b07346fbb9bcf264923cbf71ab8a7eac8cc713fd37f511f4c32 WatchSource:0}: Error finding container dcd687a25da90b07346fbb9bcf264923cbf71ab8a7eac8cc713fd37f511f4c32: Status 404 returned error can't find the container with id dcd687a25da90b07346fbb9bcf264923cbf71ab8a7eac8cc713fd37f511f4c32
Sep 18 18:40:12.106833 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.106809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"]
Sep 18 18:40:12.109423 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:12.109398    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9623c6a_3a5b_452b_a8e2_a437692b9a98.slice/crio-6c513000c396bbdd4e4d673915775d91e57796b605aef61636c45d47c8f1786b WatchSource:0}: Error finding container 6c513000c396bbdd4e4d673915775d91e57796b605aef61636c45d47c8f1786b: Status 404 returned error can't find the container with id 6c513000c396bbdd4e4d673915775d91e57796b605aef61636c45d47c8f1786b
Sep 18 18:40:12.158048 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.158019    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687dbbd5d8-hvg4r"]
Sep 18 18:40:12.161186 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:12.161153    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872f73e0_7d28_4967_8473_6276d1c7ebbd.slice/crio-ca68af2e8adcab6bfddb34b537d33688ffdc38400fb889d7b904d7a668cf8884 WatchSource:0}: Error finding container ca68af2e8adcab6bfddb34b537d33688ffdc38400fb889d7b904d7a668cf8884: Status 404 returned error can't find the container with id ca68af2e8adcab6bfddb34b537d33688ffdc38400fb889d7b904d7a668cf8884
Sep 18 18:40:12.877330 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.877223    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz" event={"ID":"047b794b-3638-44f6-a79f-610ca0a5736d","Type":"ContainerStarted","Data":"dcd687a25da90b07346fbb9bcf264923cbf71ab8a7eac8cc713fd37f511f4c32"}
Sep 18 18:40:12.879123 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.879087    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg" event={"ID":"a9623c6a-3a5b-452b-a8e2-a437692b9a98","Type":"ContainerStarted","Data":"6c513000c396bbdd4e4d673915775d91e57796b605aef61636c45d47c8f1786b"}
Sep 18 18:40:12.880877 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.880853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r" event={"ID":"872f73e0-7d28-4967-8473-6276d1c7ebbd","Type":"ContainerStarted","Data":"bceef5ff624848688dc7d2248edfdbef2f46436da2ca68f4a6f62291d8241625"}
Sep 18 18:40:12.881019 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.880919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r" event={"ID":"872f73e0-7d28-4967-8473-6276d1c7ebbd","Type":"ContainerStarted","Data":"ca68af2e8adcab6bfddb34b537d33688ffdc38400fb889d7b904d7a668cf8884"}
Sep 18 18:40:12.881661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.881642    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:12.885038 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.885013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj" event={"ID":"72574134-252d-44ec-bc3b-7bd3bbb22350","Type":"ContainerStarted","Data":"766e2e9445eaec3a4597d51b2a8dca230e6f483a79661fad7684eadee4c50582"}
Sep 18 18:40:12.900741 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:12.900689    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r" podStartSLOduration=1.900675087 podStartE2EDuration="1.900675087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:12.899554277 +0000 UTC m=+55.919350991" watchObservedRunningTime="2026-09-18 18:40:12.900675087 +0000 UTC m=+55.920471712"
Sep 18 18:40:13.851413 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:13.851246    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nxqvl"
Sep 18 18:40:18.785124 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.785095    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zl87"
Sep 18 18:40:18.910283 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.910218    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg" event={"ID":"a9623c6a-3a5b-452b-a8e2-a437692b9a98","Type":"ContainerStarted","Data":"dc75e04b2fa8110817a4bb8d9afee03a610c90db7c8745096436157a79f9ccf4"}
Sep 18 18:40:18.910514 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.910493    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:18.911740 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.911695    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj" event={"ID":"72574134-252d-44ec-bc3b-7bd3bbb22350","Type":"ContainerStarted","Data":"2e3c504b371c570311604596f4d869b2910caa8e4ff5dce52753964565ca5673"}
Sep 18 18:40:18.912199 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.912184    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg"
Sep 18 18:40:18.913640 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.913622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz" event={"ID":"047b794b-3638-44f6-a79f-610ca0a5736d","Type":"ContainerStarted","Data":"7a76f9e47ad878a0d5861034cb71b105c985b3fc28b75df10797332da2c2e142"}
Sep 18 18:40:18.913728 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.913654    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz" event={"ID":"047b794b-3638-44f6-a79f-610ca0a5736d","Type":"ContainerStarted","Data":"d05ab1a569ce3e59377b591d6e4e58ec05e1b71ec1409b7b3fbfa17533e529ca"}
Sep 18 18:40:18.927696 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.927656    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64665b9646-ggbzg" podStartSLOduration=1.2996217749999999 podStartE2EDuration="7.927641332s" totalImagesPullingTime="6.628019557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:18.927174195 +0000 UTC m=+61.946970821" watchObservedRunningTime="2026-09-18 18:40:18.927641332 +0000 UTC m=+61.947437958"
Sep 18 18:40:18.941271 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:18.941214    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64bb6c5996-bqlxj" podStartSLOduration=1.30595869 podStartE2EDuration="7.941195217s" totalImagesPullingTime="6.635236527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:18.94065528 +0000 UTC m=+61.960451903" watchObservedRunningTime="2026-09-18 18:40:18.941195217 +0000 UTC m=+61.960991846"
Sep 18 18:40:19.918789 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:19.918752    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz" event={"ID":"047b794b-3638-44f6-a79f-610ca0a5736d","Type":"ContainerStarted","Data":"f134e5b75e7799e043eb66bee364119fe5c367aaf6f1faf68cef79f010553504"}
Sep 18 18:40:19.936830 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:19.936780    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6fd9964594-b5wdz" podStartSLOduration=2.301371694 podStartE2EDuration="8.936763798s" totalImagesPullingTime="6.635392104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:19.936160066 +0000 UTC m=+62.955956693" watchObservedRunningTime="2026-09-18 18:40:19.936763798 +0000 UTC m=+62.956560418"
Sep 18 18:40:21.494233 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:21.494207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:22.245570 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.245531    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:40:22.245745 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.245591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:40:22.247862 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.247845    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:40:22.248534 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.248521    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:40:22.258610 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.258581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244d630b-d03b-4e98-8f83-81a041d5eaee-original-pull-secret\") pod \"global-pull-secret-syncer-5htxl\" (UID: \"244d630b-d03b-4e98-8f83-81a041d5eaee\") " pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:40:22.258716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.258612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bdb7a9a-d783-47ac-aeb1-92b826817290-metrics-certs\") pod \"network-metrics-daemon-g2gmk\" (UID: \"9bdb7a9a-d783-47ac-aeb1-92b826817290\") " pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:40:22.346280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.346243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:40:22.348689 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.348669    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:40:22.359381 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.359365    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:40:22.370575 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.370546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkfh\" (UniqueName: \"kubernetes.io/projected/a59d9eb2-9eb0-42c7-b656-49679e3b164b-kube-api-access-nrkfh\") pod \"network-check-target-6frvx\" (UID: \"a59d9eb2-9eb0-42c7-b656-49679e3b164b\") " pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:40:22.441679 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.441650    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kbgtj\""
Sep 18 18:40:22.447765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.447745    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mb7g\""
Sep 18 18:40:22.449855 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.449841    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:40:22.452616 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.452593    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5htxl"
Sep 18 18:40:22.456128 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.456109    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2gmk"
Sep 18 18:40:22.731565 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.731536    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6frvx"]
Sep 18 18:40:22.734827 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:22.734800    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59d9eb2_9eb0_42c7_b656_49679e3b164b.slice/crio-58faf5aa7099eaaf6502213aee010fc610bf745005fcbf8bdf67a690456f79c6 WatchSource:0}: Error finding container 58faf5aa7099eaaf6502213aee010fc610bf745005fcbf8bdf67a690456f79c6: Status 404 returned error can't find the container with id 58faf5aa7099eaaf6502213aee010fc610bf745005fcbf8bdf67a690456f79c6
Sep 18 18:40:22.736811 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.736788    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5htxl"]
Sep 18 18:40:22.741105 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.741080    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2gmk"]
Sep 18 18:40:22.741807 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:22.741717    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244d630b_d03b_4e98_8f83_81a041d5eaee.slice/crio-0758f1a84480c717ca3e6f62d25304db51230edf173bd9b4baa8557fffac89cf WatchSource:0}: Error finding container 0758f1a84480c717ca3e6f62d25304db51230edf173bd9b4baa8557fffac89cf: Status 404 returned error can't find the container with id 0758f1a84480c717ca3e6f62d25304db51230edf173bd9b4baa8557fffac89cf
Sep 18 18:40:22.743872 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:22.743848    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdb7a9a_d783_47ac_aeb1_92b826817290.slice/crio-5bddad0bd0d906b7e48dc7fda8833c20b6c976775b53e29ef37eb704c7634771 WatchSource:0}: Error finding container 5bddad0bd0d906b7e48dc7fda8833c20b6c976775b53e29ef37eb704c7634771: Status 404 returned error can't find the container with id 5bddad0bd0d906b7e48dc7fda8833c20b6c976775b53e29ef37eb704c7634771
Sep 18 18:40:22.929511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.929477    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5htxl" event={"ID":"244d630b-d03b-4e98-8f83-81a041d5eaee","Type":"ContainerStarted","Data":"0758f1a84480c717ca3e6f62d25304db51230edf173bd9b4baa8557fffac89cf"}
Sep 18 18:40:22.930539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.930516    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6frvx" event={"ID":"a59d9eb2-9eb0-42c7-b656-49679e3b164b","Type":"ContainerStarted","Data":"58faf5aa7099eaaf6502213aee010fc610bf745005fcbf8bdf67a690456f79c6"}
Sep 18 18:40:22.931645 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:22.931624    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2gmk" event={"ID":"9bdb7a9a-d783-47ac-aeb1-92b826817290","Type":"ContainerStarted","Data":"5bddad0bd0d906b7e48dc7fda8833c20b6c976775b53e29ef37eb704c7634771"}
Sep 18 18:40:24.940362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:24.940326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2gmk" event={"ID":"9bdb7a9a-d783-47ac-aeb1-92b826817290","Type":"ContainerStarted","Data":"65806383cc89862660e175a5f219b0c9f67280d25fae861323a264195c288933"}
Sep 18 18:40:24.940793 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:24.940379    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2gmk" event={"ID":"9bdb7a9a-d783-47ac-aeb1-92b826817290","Type":"ContainerStarted","Data":"764c1dc9441cb73fd2793a2e1eabd16ea0c13d36e91ad3977642fcc547b45df0"}
Sep 18 18:40:24.955971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:24.955628    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g2gmk" podStartSLOduration=66.566510166 podStartE2EDuration="1m7.955604218s" totalImagesPullingTime="1.389094052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:24.954488124 +0000 UTC m=+67.974284752" watchObservedRunningTime="2026-09-18 18:40:24.955604218 +0000 UTC m=+67.975400850"
Sep 18 18:40:27.953523 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:27.953436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5htxl" event={"ID":"244d630b-d03b-4e98-8f83-81a041d5eaee","Type":"ContainerStarted","Data":"97e4d10aa2ee6b7b0281e14d13d4ce5944209e60bcdca9c67d0031a168fff984"}
Sep 18 18:40:27.955009 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:27.954983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6frvx" event={"ID":"a59d9eb2-9eb0-42c7-b656-49679e3b164b","Type":"ContainerStarted","Data":"7bb3fced3a4ec5b45b59e1e0787a5f2c785a1e364ce443128d1597fcb69a625e"}
Sep 18 18:40:27.955142 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:27.955116    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:40:27.967552 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:27.967510    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5htxl" podStartSLOduration=66.120954962 podStartE2EDuration="1m10.967496775s" totalImagesPullingTime="4.846541813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:27.966855497 +0000 UTC m=+70.986652123" watchObservedRunningTime="2026-09-18 18:40:27.967496775 +0000 UTC m=+70.987293401"
Sep 18 18:40:27.980429 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:27.980386    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6frvx" podStartSLOduration=66.132464171 podStartE2EDuration="1m10.980372825s" totalImagesPullingTime="4.847908654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:27.979728924 +0000 UTC m=+70.999525550" watchObservedRunningTime="2026-09-18 18:40:27.980372825 +0000 UTC m=+71.000169448"
Sep 18 18:40:29.477999 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.477965    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mh2rr"]
Sep 18 18:40:29.480190 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.480168    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.483303 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.483276    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:40:29.483553 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.483537    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:40:29.483610 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.483539    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s6ph4\""
Sep 18 18:40:29.484310 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.484292    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:40:29.484310 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.484298    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:40:29.484440 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.484300    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:40:29.484620 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.484604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:40:29.503343 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503376    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-metrics-client-ca\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503434    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm6xl\" (UniqueName: \"kubernetes.io/projected/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-kube-api-access-tm6xl\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-wtmp\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-root\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503576    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-textfile\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.503661 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.503595    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-sys\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604260 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604260 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-metrics-client-ca\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm6xl\" (UniqueName: \"kubernetes.io/projected/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-kube-api-access-tm6xl\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-wtmp\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604355    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-root\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-textfile\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-sys\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604524 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-wtmp\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604932 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:40:29.604549    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 18:40:29.604932 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:40:29.604604    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls podName:0f81b2ec-5918-4a0a-8eeb-440c5dff2d34 nodeName:}" failed. No retries permitted until 2026-09-18 18:40:30.104585196 +0000 UTC m=+73.124381805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls") pod "node-exporter-mh2rr" (UID: "0f81b2ec-5918-4a0a-8eeb-440c5dff2d34") : secret "node-exporter-tls" not found
Sep 18 18:40:29.604932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-sys\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-root\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.604932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604764    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-textfile\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.605131 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-metrics-client-ca\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.605131 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.604991    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.606864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.606837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:29.613290 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:29.613264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm6xl\" (UniqueName: \"kubernetes.io/projected/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-kube-api-access-tm6xl\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:30.108209 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:30.108167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:30.110659 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:30.110624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f81b2ec-5918-4a0a-8eeb-440c5dff2d34-node-exporter-tls\") pod \"node-exporter-mh2rr\" (UID: \"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34\") " pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:30.389077 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:30.389001    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh2rr"
Sep 18 18:40:30.400843 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:40:30.400806    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f81b2ec_5918_4a0a_8eeb_440c5dff2d34.slice/crio-0520556d9212b5fd8457cad5090b3425190e0f80c3a20788b6318f5334097d0d WatchSource:0}: Error finding container 0520556d9212b5fd8457cad5090b3425190e0f80c3a20788b6318f5334097d0d: Status 404 returned error can't find the container with id 0520556d9212b5fd8457cad5090b3425190e0f80c3a20788b6318f5334097d0d
Sep 18 18:40:30.972401 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:30.972365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh2rr" event={"ID":"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34","Type":"ContainerStarted","Data":"0520556d9212b5fd8457cad5090b3425190e0f80c3a20788b6318f5334097d0d"}
Sep 18 18:40:31.977938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:31.977882    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f81b2ec-5918-4a0a-8eeb-440c5dff2d34" containerID="2cfcdcea1b47f57a9fdf4dd9894d10c567972fb52be76c0cb7e9dd9c03cc8ce1" exitCode=0
Sep 18 18:40:31.977938 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:31.977925    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh2rr" event={"ID":"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34","Type":"ContainerDied","Data":"2cfcdcea1b47f57a9fdf4dd9894d10c567972fb52be76c0cb7e9dd9c03cc8ce1"}
Sep 18 18:40:32.984472 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:32.984436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh2rr" event={"ID":"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34","Type":"ContainerStarted","Data":"b3ce14adb2344a4565e25447f5292206cc5706d7a4db27f2c9d55867c28290d3"}
Sep 18 18:40:32.984930 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:32.984491    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh2rr" event={"ID":"0f81b2ec-5918-4a0a-8eeb-440c5dff2d34","Type":"ContainerStarted","Data":"d7254c675c4cbe5a9d5ce17a98c7d8e7336155dfe77073eee253aa320d2bdb3f"}
Sep 18 18:40:33.008210 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:33.008166    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mh2rr" podStartSLOduration=2.976629811 podStartE2EDuration="4.008148415s" totalImagesPullingTime="960.740598ms" totalInitContainerRuntime="70.778006ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:33.006334271 +0000 UTC m=+76.026130896" watchObservedRunningTime="2026-09-18 18:40:33.008148415 +0000 UTC m=+76.027945041"
Sep 18 18:40:34.898634 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:34.898608    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-687dbbd5d8-hvg4r"
Sep 18 18:40:36.509377 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.509330    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" podUID="962d2637-c781-468b-9bbb-bc4ed05342f8" containerName="registry" containerID="cri-o://6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89" gracePeriod=30
Sep 18 18:40:36.754045 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.754011    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:36.862140 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862099    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862153    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862187    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862449 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862375    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862507 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862447    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc8h6\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862507 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862477    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862507    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862539    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets\") pod \"962d2637-c781-468b-9bbb-bc4ed05342f8\" (UID: \"962d2637-c781-468b-9bbb-bc4ed05342f8\") "
Sep 18 18:40:36.862763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862596    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:40:36.862763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862653    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:40:36.863119 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862805    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-trusted-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.863119 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.862823    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-certificates\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.865393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.865360    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:40:36.865393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.865367    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:40:36.865544 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.865481    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6" (OuterVolumeSpecName: "kube-api-access-bc8h6") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "kube-api-access-bc8h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:40:36.865544 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.865492    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:40:36.865612 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.865555    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:40:36.871323 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.871293    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "962d2637-c781-468b-9bbb-bc4ed05342f8" (UID: "962d2637-c781-468b-9bbb-bc4ed05342f8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:40:36.964091 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964053    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-registry-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.964091 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964087    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-image-registry-private-configuration\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.964091 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964098    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc8h6\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-kube-api-access-bc8h6\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.964320 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964110    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/962d2637-c781-468b-9bbb-bc4ed05342f8-ca-trust-extracted\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.964320 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964121    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/962d2637-c781-468b-9bbb-bc4ed05342f8-bound-sa-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:36.964320 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:36.964129    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/962d2637-c781-468b-9bbb-bc4ed05342f8-installation-pull-secrets\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:40:37.004139 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.004105    2558 generic.go:356] "Generic (PLEG): container finished" podID="962d2637-c781-468b-9bbb-bc4ed05342f8" containerID="6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89" exitCode=0
Sep 18 18:40:37.004333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.004151    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6"
Sep 18 18:40:37.004333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.004198    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" event={"ID":"962d2637-c781-468b-9bbb-bc4ed05342f8","Type":"ContainerDied","Data":"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"}
Sep 18 18:40:37.004333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.004257    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bb87766b-lbcz6" event={"ID":"962d2637-c781-468b-9bbb-bc4ed05342f8","Type":"ContainerDied","Data":"233830082a46ac8a99282cf3b5a77277fa173febbf89abdf4fb6b14584004c4a"}
Sep 18 18:40:37.004333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.004284    2558 scope.go:117] "RemoveContainer" containerID="6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"
Sep 18 18:40:37.013877 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.013856    2558 scope.go:117] "RemoveContainer" containerID="6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"
Sep 18 18:40:37.014280 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:40:37.014258    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89\": container with ID starting with 6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89 not found: ID does not exist" containerID="6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"
Sep 18 18:40:37.014328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.014289    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89"} err="failed to get container status \"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89\": rpc error: code = NotFound desc = could not find container \"6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89\": container with ID starting with 6b2dc955ed93f85c5dd8c9cbf7436d5bc78c25f4ed2717330a5f50be46c48a89 not found: ID does not exist"
Sep 18 18:40:37.025655 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.025619    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:40:37.031914 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.031852    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8bb87766b-lbcz6"]
Sep 18 18:40:37.433621 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:37.433586    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962d2637-c781-468b-9bbb-bc4ed05342f8" path="/var/lib/kubelet/pods/962d2637-c781-468b-9bbb-bc4ed05342f8/volumes"
Sep 18 18:40:58.960833 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:40:58.960798    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6frvx"
Sep 18 18:44:17.330863 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:44:17.330837    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:45:22.870159 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.870126    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"]
Sep 18 18:45:22.870565 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.870367    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962d2637-c781-468b-9bbb-bc4ed05342f8" containerName="registry"
Sep 18 18:45:22.870565 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.870379    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="962d2637-c781-468b-9bbb-bc4ed05342f8" containerName="registry"
Sep 18 18:45:22.870565 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.870429    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="962d2637-c781-468b-9bbb-bc4ed05342f8" containerName="registry"
Sep 18 18:45:22.872138 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.872121    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:22.876726 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.876701    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:45:22.876858 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.876749    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:45:22.876996 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.876976    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fnn27\""
Sep 18 18:45:22.880058 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.880040    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:45:22.893632 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.893607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:45:22.896443 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.896421    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"]
Sep 18 18:45:22.928300 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.928267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:22.928474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.928315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8v97\" (UniqueName: \"kubernetes.io/projected/60843eb3-bc5e-4be0-8b85-0e3883447d78-kube-api-access-s8v97\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:22.928474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.928342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:22.928474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.928407    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:22.928474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:22.928427    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029608 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029608 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8v97\" (UniqueName: \"kubernetes.io/projected/60843eb3-bc5e-4be0-8b85-0e3883447d78-kube-api-access-s8v97\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029608 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.029849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.029812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.031883 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.031853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60843eb3-bc5e-4be0-8b85-0e3883447d78-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.032020 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.032003    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.032096 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.032079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60843eb3-bc5e-4be0-8b85-0e3883447d78-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.038668 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.038643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8v97\" (UniqueName: \"kubernetes.io/projected/60843eb3-bc5e-4be0-8b85-0e3883447d78-kube-api-access-s8v97\") pod \"opendatahub-operator-controller-manager-55bcff78dd-rjx49\" (UID: \"60843eb3-bc5e-4be0-8b85-0e3883447d78\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.182020 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.181933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:23.457245 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.457166    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"]
Sep 18 18:45:23.460565 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:45:23.460534    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60843eb3_bc5e_4be0_8b85_0e3883447d78.slice/crio-0e9ecb34cebb4bbd8e3801ad95ccc4d5f15580a706ae7e2d8ef23aed0906dce0 WatchSource:0}: Error finding container 0e9ecb34cebb4bbd8e3801ad95ccc4d5f15580a706ae7e2d8ef23aed0906dce0: Status 404 returned error can't find the container with id 0e9ecb34cebb4bbd8e3801ad95ccc4d5f15580a706ae7e2d8ef23aed0906dce0
Sep 18 18:45:23.462367 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.462352    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:45:23.952782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:23.952747    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49" event={"ID":"60843eb3-bc5e-4be0-8b85-0e3883447d78","Type":"ContainerStarted","Data":"0e9ecb34cebb4bbd8e3801ad95ccc4d5f15580a706ae7e2d8ef23aed0906dce0"}
Sep 18 18:45:26.968567 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:26.968483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49" event={"ID":"60843eb3-bc5e-4be0-8b85-0e3883447d78","Type":"ContainerStarted","Data":"8ccfe4833b10073e2c958ba509d97e81440a922b62a41ba845689a2e4793ac3c"}
Sep 18 18:45:27.973075 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:27.972973    2558 generic.go:356] "Generic (PLEG): container finished" podID="60843eb3-bc5e-4be0-8b85-0e3883447d78" containerID="8ccfe4833b10073e2c958ba509d97e81440a922b62a41ba845689a2e4793ac3c" exitCode=0
Sep 18 18:45:27.973547 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:27.973090    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49" event={"ID":"60843eb3-bc5e-4be0-8b85-0e3883447d78","Type":"ContainerDied","Data":"8ccfe4833b10073e2c958ba509d97e81440a922b62a41ba845689a2e4793ac3c"}
Sep 18 18:45:28.977767 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:28.977735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49" event={"ID":"60843eb3-bc5e-4be0-8b85-0e3883447d78","Type":"ContainerStarted","Data":"63a9fa17bc51cddb6ca6be901f9e00ca2496eea8498fcc20647fb43e7df38272"}
Sep 18 18:45:28.978168 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:28.977853    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:29.018695 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:29.018633    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49" podStartSLOduration=2.5046461989999997 podStartE2EDuration="7.01860944s" totalImagesPullingTime="3.519443216s" totalInitContainerRuntime="994.520025ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:29.01594471 +0000 UTC m=+372.035741337" watchObservedRunningTime="2026-09-18 18:45:29.01860944 +0000 UTC m=+372.038406067"
Sep 18 18:45:39.982348 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:39.982317    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-rjx49"
Sep 18 18:45:42.588602 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.588569    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"]
Sep 18 18:45:42.593513 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.593484    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.603732 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.603710    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 18:45:42.604813 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.604798    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:45:42.605131 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.605116    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:45:42.605212 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.605167    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:45:42.610611 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.610586    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"]
Sep 18 18:45:42.675539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.675502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf2c4836-df42-44a7-a965-3ec85386f45e-tmp\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.675539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.675540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.675763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.675582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrq7\" (UniqueName: \"kubernetes.io/projected/cf2c4836-df42-44a7-a965-3ec85386f45e-kube-api-access-bvrq7\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.777033 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.776989    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrq7\" (UniqueName: \"kubernetes.io/projected/cf2c4836-df42-44a7-a965-3ec85386f45e-kube-api-access-bvrq7\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.777234 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.777056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf2c4836-df42-44a7-a965-3ec85386f45e-tmp\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.777234 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.777116    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.777363 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:42.777241    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 18:45:42.777363 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:42.777343    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs podName:cf2c4836-df42-44a7-a965-3ec85386f45e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:43.277325274 +0000 UTC m=+386.297121878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs") pod "kube-auth-proxy-c66559bbb-6lzls" (UID: "cf2c4836-df42-44a7-a965-3ec85386f45e") : secret "kube-auth-proxy-tls" not found
Sep 18 18:45:42.779539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.779508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf2c4836-df42-44a7-a965-3ec85386f45e-tmp\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:42.786376 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:42.786349    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrq7\" (UniqueName: \"kubernetes.io/projected/cf2c4836-df42-44a7-a965-3ec85386f45e-kube-api-access-bvrq7\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:43.282554 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.282502    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:43.285151 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.285122    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2c4836-df42-44a7-a965-3ec85386f45e-tls-certs\") pod \"kube-auth-proxy-c66559bbb-6lzls\" (UID: \"cf2c4836-df42-44a7-a965-3ec85386f45e\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:43.502445 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.502402    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"
Sep 18 18:45:43.773959 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.773923    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-6lzls"]
Sep 18 18:45:43.777223 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:45:43.777197    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2c4836_df42_44a7_a965_3ec85386f45e.slice/crio-ef4a8ce80ceaa77a5a65d8881e6ca186b38f424327125ac4805e1183fa94d466 WatchSource:0}: Error finding container ef4a8ce80ceaa77a5a65d8881e6ca186b38f424327125ac4805e1183fa94d466: Status 404 returned error can't find the container with id ef4a8ce80ceaa77a5a65d8881e6ca186b38f424327125ac4805e1183fa94d466
Sep 18 18:45:43.933100 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.933024    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:45:43.935715 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.935699    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:43.938856 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.938820    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:45:43.938856 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.938827    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 18:45:43.939049 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.938915    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 18:45:43.939049 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.938844    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 18:45:43.939604 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.939589    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qmjgm\""
Sep 18 18:45:43.954912 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:43.954862    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:45:44.030714 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.030675    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls" event={"ID":"cf2c4836-df42-44a7-a965-3ec85386f45e","Type":"ContainerStarted","Data":"ef4a8ce80ceaa77a5a65d8881e6ca186b38f424327125ac4805e1183fa94d466"}
Sep 18 18:45:44.090111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090118    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090184    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmcls\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090277    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.090328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.090300    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191465 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191374    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191465 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191465 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmcls\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191706 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191706 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191706 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191670    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.191835 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.191722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.192216 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.192166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.193981 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.193959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.194070 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.193960    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.194411 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.194391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.194505 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.194419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.199692 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.199669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.200208 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.200178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmcls\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls\") pod \"istiod-openshift-gateway-7b7995d5cd-4lccv\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.245086 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.245054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:44.517535 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:44.517028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:45:44.520844 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:45:44.520814    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970b3e40_4846_4c0e_ad62_2fae72fb19db.slice/crio-dfa3ae362ad22283e4d1b0569796cbf96d6fe327560e6ab67550cd52180281b4 WatchSource:0}: Error finding container dfa3ae362ad22283e4d1b0569796cbf96d6fe327560e6ab67550cd52180281b4: Status 404 returned error can't find the container with id dfa3ae362ad22283e4d1b0569796cbf96d6fe327560e6ab67550cd52180281b4
Sep 18 18:45:45.036371 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:45.036334    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" event={"ID":"970b3e40-4846-4c0e-ad62-2fae72fb19db","Type":"ContainerStarted","Data":"dfa3ae362ad22283e4d1b0569796cbf96d6fe327560e6ab67550cd52180281b4"}
Sep 18 18:45:47.937529 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:47.937475    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:45:47.937954 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:47.937560    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:45:48.050146 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:48.050108    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" event={"ID":"970b3e40-4846-4c0e-ad62-2fae72fb19db","Type":"ContainerStarted","Data":"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"}
Sep 18 18:45:48.050347 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:48.050211    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:48.051786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:48.051761    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls" event={"ID":"cf2c4836-df42-44a7-a965-3ec85386f45e","Type":"ContainerStarted","Data":"bc08bcd875fbf2f077de9cdf6774026014d35675c54d249ca0f4e2f04f1a583a"}
Sep 18 18:45:48.071104 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:48.071051    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" podStartSLOduration=1.65760188 podStartE2EDuration="5.071036286s" totalImagesPullingTime="3.413434406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:48.06917276 +0000 UTC m=+391.088969388" watchObservedRunningTime="2026-09-18 18:45:48.071036286 +0000 UTC m=+391.090832912"
Sep 18 18:45:48.086990 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:48.086937    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c66559bbb-6lzls" podStartSLOduration=1.9260811599999998 podStartE2EDuration="6.086917379s" totalImagesPullingTime="4.160836219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:48.085975488 +0000 UTC m=+391.105772114" watchObservedRunningTime="2026-09-18 18:45:48.086917379 +0000 UTC m=+391.106714012"
Sep 18 18:45:49.058198 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:49.058160    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-4lccv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:45:49.058582 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:49.058219    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:45:52.057663 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.057625    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:45:52.915491 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.915457    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fshmm"]
Sep 18 18:45:52.919640 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.919621    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:52.922053 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.922023    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 18:45:52.922201 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.922164    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cxrfv\""
Sep 18 18:45:52.930218 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.930143    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fshmm"]
Sep 18 18:45:52.962736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.962701    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj8d5\" (UniqueName: \"kubernetes.io/projected/9afc7b95-1779-4d18-b608-b99bca4388a0-kube-api-access-tj8d5\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:52.962876 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:52.962752    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.064002 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.063967    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj8d5\" (UniqueName: \"kubernetes.io/projected/9afc7b95-1779-4d18-b608-b99bca4388a0-kube-api-access-tj8d5\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.064393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.064009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.064393 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:53.064118    2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 18 18:45:53.064393 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:53.064188    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert podName:9afc7b95-1779-4d18-b608-b99bca4388a0 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:53.564166936 +0000 UTC m=+396.583963539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert") pod "odh-model-controller-7998c67dc4-fshmm" (UID: "9afc7b95-1779-4d18-b608-b99bca4388a0") : secret "odh-model-controller-webhook-cert" not found
Sep 18 18:45:53.074201 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.074167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj8d5\" (UniqueName: \"kubernetes.io/projected/9afc7b95-1779-4d18-b608-b99bca4388a0-kube-api-access-tj8d5\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.279066 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.279031    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8jhwg"]
Sep 18 18:45:53.282365 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.282350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.284539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.284518    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 18:45:53.284539 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.284529    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qg9km\""
Sep 18 18:45:53.291692 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.291663    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8jhwg"]
Sep 18 18:45:53.366570 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.366531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvpxz\" (UniqueName: \"kubernetes.io/projected/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-kube-api-access-gvpxz\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.366731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.366619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.467188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.467147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.467379 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.467251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvpxz\" (UniqueName: \"kubernetes.io/projected/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-kube-api-access-gvpxz\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.467379 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:53.467292    2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 18 18:45:53.467490 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:45:53.467382    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs podName:c46fce34-bd2a-4c7b-9e21-e837fc3595c9 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:53.967366692 +0000 UTC m=+396.987163318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs") pod "model-serving-api-54d89bd5fc-8jhwg" (UID: "c46fce34-bd2a-4c7b-9e21-e837fc3595c9") : secret "model-serving-api-tls" not found
Sep 18 18:45:53.478601 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.478574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvpxz\" (UniqueName: \"kubernetes.io/projected/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-kube-api-access-gvpxz\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.568475 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.568388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.570951 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.570924    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9afc7b95-1779-4d18-b608-b99bca4388a0-cert\") pod \"odh-model-controller-7998c67dc4-fshmm\" (UID: \"9afc7b95-1779-4d18-b608-b99bca4388a0\") " pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.831775 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.831678    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:53.971875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.971833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:53.974519 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:53.974494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c46fce34-bd2a-4c7b-9e21-e837fc3595c9-tls-certs\") pod \"model-serving-api-54d89bd5fc-8jhwg\" (UID: \"c46fce34-bd2a-4c7b-9e21-e837fc3595c9\") " pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:54.101747 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:54.101649    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fshmm"]
Sep 18 18:45:54.106130 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:45:54.106097    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afc7b95_1779_4d18_b608_b99bca4388a0.slice/crio-9cb77af0d3bd2dbeac5ddf20a8d17341629e0206405eb618de81010265f92d85 WatchSource:0}: Error finding container 9cb77af0d3bd2dbeac5ddf20a8d17341629e0206405eb618de81010265f92d85: Status 404 returned error can't find the container with id 9cb77af0d3bd2dbeac5ddf20a8d17341629e0206405eb618de81010265f92d85
Sep 18 18:45:54.192994 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:54.192957    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:54.469591 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:54.469512    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8jhwg"]
Sep 18 18:45:54.472337 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:45:54.472309    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46fce34_bd2a_4c7b_9e21_e837fc3595c9.slice/crio-e17e12042aae879d151dd5f3027850d5f946cc4e18474f9f885dbcab3fbc4157 WatchSource:0}: Error finding container e17e12042aae879d151dd5f3027850d5f946cc4e18474f9f885dbcab3fbc4157: Status 404 returned error can't find the container with id e17e12042aae879d151dd5f3027850d5f946cc4e18474f9f885dbcab3fbc4157
Sep 18 18:45:55.080529 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:55.080488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg" event={"ID":"c46fce34-bd2a-4c7b-9e21-e837fc3595c9","Type":"ContainerStarted","Data":"e17e12042aae879d151dd5f3027850d5f946cc4e18474f9f885dbcab3fbc4157"}
Sep 18 18:45:55.081873 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:55.081829    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm" event={"ID":"9afc7b95-1779-4d18-b608-b99bca4388a0","Type":"ContainerStarted","Data":"9cb77af0d3bd2dbeac5ddf20a8d17341629e0206405eb618de81010265f92d85"}
Sep 18 18:45:58.097094 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.097047    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg" event={"ID":"c46fce34-bd2a-4c7b-9e21-e837fc3595c9","Type":"ContainerStarted","Data":"6738dfccf85eab9ceb384bf03a4c7e861c861bd0712930cea69329516b235fe5"}
Sep 18 18:45:58.097637 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.097158    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:45:58.098762 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.098740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm" event={"ID":"9afc7b95-1779-4d18-b608-b99bca4388a0","Type":"ContainerStarted","Data":"aadc7619e2e1e7827cd56ab2015d918b5b4424b6752b50755d60719fe5d49a7a"}
Sep 18 18:45:58.098880 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.098866    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:45:58.114315 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.114241    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg" podStartSLOduration=2.208098674 podStartE2EDuration="5.114219044s" totalImagesPullingTime="2.90612037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:58.113601599 +0000 UTC m=+401.133398236" watchObservedRunningTime="2026-09-18 18:45:58.114219044 +0000 UTC m=+401.134015705"
Sep 18 18:45:58.131530 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:45:58.131472    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm" podStartSLOduration=2.8548433490000003 podStartE2EDuration="6.131451057s" totalImagesPullingTime="3.276607708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:58.129281328 +0000 UTC m=+401.149077976" watchObservedRunningTime="2026-09-18 18:45:58.131451057 +0000 UTC m=+401.151247685"
Sep 18 18:46:01.095911 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.095847    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"]
Sep 18 18:46:01.099140 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.099120    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.116360 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.116332    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"]
Sep 18 18:46:01.139510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de0c6b90-a1ab-467f-ae6d-c9b435756071-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td57v\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-kube-api-access-td57v\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.139731 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.139695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240616 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240616 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de0c6b90-a1ab-467f-ae6d-c9b435756071-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.240852 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.240770    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td57v\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-kube-api-access-td57v\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.241799 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.241765    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.243269 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.243239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.243383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.243335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de0c6b90-a1ab-467f-ae6d-c9b435756071-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.243443 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.243401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.243488 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.243444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.249354 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.249322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td57v\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-kube-api-access-td57v\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.250376 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.250353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de0c6b90-a1ab-467f-ae6d-c9b435756071-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tzknv\" (UID: \"de0c6b90-a1ab-467f-ae6d-c9b435756071\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.408761 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.408668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:01.693864 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.693647    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"]
Sep 18 18:46:01.696328 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:46:01.696306    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0c6b90_a1ab_467f_ae6d_c9b435756071.slice/crio-6b78512f8e942001ab881f727cc3d9e5aef8a78f332bd69f011fddaa038fb054 WatchSource:0}: Error finding container 6b78512f8e942001ab881f727cc3d9e5aef8a78f332bd69f011fddaa038fb054: Status 404 returned error can't find the container with id 6b78512f8e942001ab881f727cc3d9e5aef8a78f332bd69f011fddaa038fb054
Sep 18 18:46:01.698429 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.698400    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:46:01.698500 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:01.698458    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:46:02.119828 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:02.119781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv" event={"ID":"de0c6b90-a1ab-467f-ae6d-c9b435756071","Type":"ContainerStarted","Data":"502bd364fbc689309bd9f4b1c59ac92739aa17dfaab3f414aaf8d879309cabf3"}
Sep 18 18:46:02.120326 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:02.119875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv" event={"ID":"de0c6b90-a1ab-467f-ae6d-c9b435756071","Type":"ContainerStarted","Data":"6b78512f8e942001ab881f727cc3d9e5aef8a78f332bd69f011fddaa038fb054"}
Sep 18 18:46:02.120326 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:02.120176    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:02.122137 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:02.122104    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-tzknv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:46:02.122277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:02.122150    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv" podUID="de0c6b90-a1ab-467f-ae6d-c9b435756071" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:46:03.123624 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.123597    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv"
Sep 18 18:46:03.166350 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.166292    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tzknv" podStartSLOduration=2.166275853 podStartE2EDuration="2.166275853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:02.149763457 +0000 UTC m=+405.169560119" watchObservedRunningTime="2026-09-18 18:46:03.166275853 +0000 UTC m=+406.186072478"
Sep 18 18:46:03.263717 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.263679    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:46:03.263928 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.263885    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerName="discovery" containerID="cri-o://42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c" gracePeriod=30
Sep 18 18:46:03.499656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.499628    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:46:03.560551 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560515    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560558    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmcls\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560582    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560607    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560651    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560670    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.560716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.560688    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert\") pod \"970b3e40-4846-4c0e-ad62-2fae72fb19db\" (UID: \"970b3e40-4846-4c0e-ad62-2fae72fb19db\") "
Sep 18 18:46:03.561226 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.561180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:03.563228 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563198    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:03.563345 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563304    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:03.563517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563492    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts" (OuterVolumeSpecName: "cacerts") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:03.563583 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563515    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token" (OuterVolumeSpecName: "istio-token") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:03.563583 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563567    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs" (OuterVolumeSpecName: "local-certs") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:46:03.563646 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.563611    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls" (OuterVolumeSpecName: "kube-api-access-hmcls") pod "970b3e40-4846-4c0e-ad62-2fae72fb19db" (UID: "970b3e40-4846-4c0e-ad62-2fae72fb19db"). InnerVolumeSpecName "kube-api-access-hmcls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661425    2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-cacerts\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661452    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmcls\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-kube-api-access-hmcls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661464    2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-kubeconfig\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661473    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-ca-configmap\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661482    2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/970b3e40-4846-4c0e-ad62-2fae72fb19db-local-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661490    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.661510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.661497    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/970b3e40-4846-4c0e-ad62-2fae72fb19db-istio-csr-dns-cert\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:46:03.961966 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.961853    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"]
Sep 18 18:46:03.962224 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.962207    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerName="discovery"
Sep 18 18:46:03.962297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.962228    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerName="discovery"
Sep 18 18:46:03.962343 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.962318    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerName="discovery"
Sep 18 18:46:03.965349 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.965326    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:03.969714 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.969686    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 18 18:46:03.969831 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.969692    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-trrjk\""
Sep 18 18:46:03.981457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:03.981431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"]
Sep 18 18:46:04.064569 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.064535    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.064745 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.064595    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpgmh\" (UniqueName: \"kubernetes.io/projected/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-kube-api-access-lpgmh\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.127995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.127963    2558 generic.go:356] "Generic (PLEG): container finished" podID="970b3e40-4846-4c0e-ad62-2fae72fb19db" containerID="42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c" exitCode=0
Sep 18 18:46:04.128423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.128013    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"
Sep 18 18:46:04.128423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.128057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" event={"ID":"970b3e40-4846-4c0e-ad62-2fae72fb19db","Type":"ContainerDied","Data":"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"}
Sep 18 18:46:04.128423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.128112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv" event={"ID":"970b3e40-4846-4c0e-ad62-2fae72fb19db","Type":"ContainerDied","Data":"dfa3ae362ad22283e4d1b0569796cbf96d6fe327560e6ab67550cd52180281b4"}
Sep 18 18:46:04.128423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.128139    2558 scope.go:117] "RemoveContainer" containerID="42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"
Sep 18 18:46:04.137022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.136998    2558 scope.go:117] "RemoveContainer" containerID="42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"
Sep 18 18:46:04.137326 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:46:04.137307    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c\": container with ID starting with 42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c not found: ID does not exist" containerID="42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"
Sep 18 18:46:04.137375 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.137338    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c"} err="failed to get container status \"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c\": rpc error: code = NotFound desc = could not find container \"42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c\": container with ID starting with 42f57e4864560cb8118e77f6f93f233425796daa41f89c1602fc6ff199b8593c not found: ID does not exist"
Sep 18 18:46:04.165398 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.165363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgmh\" (UniqueName: \"kubernetes.io/projected/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-kube-api-access-lpgmh\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.165712 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.165689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.168154 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.168136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.173916 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.173869    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:46:04.181691 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.181665    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4lccv"]
Sep 18 18:46:04.182120 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.182104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpgmh\" (UniqueName: \"kubernetes.io/projected/2e428cc9-49f8-49c4-a23d-a9f769b5bb58-kube-api-access-lpgmh\") pod \"llmisvc-controller-manager-777c6bd7d5-4687t\" (UID: \"2e428cc9-49f8-49c4-a23d-a9f769b5bb58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.276082 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.276053    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:04.565627 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:04.565600    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"]
Sep 18 18:46:04.567383 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:46:04.567348    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e428cc9_49f8_49c4_a23d_a9f769b5bb58.slice/crio-a7a6c3683d26a0e07e76fa8225b8b31ae29c56d2ffcdfbab9012381e59c82569 WatchSource:0}: Error finding container a7a6c3683d26a0e07e76fa8225b8b31ae29c56d2ffcdfbab9012381e59c82569: Status 404 returned error can't find the container with id a7a6c3683d26a0e07e76fa8225b8b31ae29c56d2ffcdfbab9012381e59c82569
Sep 18 18:46:05.135503 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:05.135464    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t" event={"ID":"2e428cc9-49f8-49c4-a23d-a9f769b5bb58","Type":"ContainerStarted","Data":"a7a6c3683d26a0e07e76fa8225b8b31ae29c56d2ffcdfbab9012381e59c82569"}
Sep 18 18:46:05.433116 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:05.433041    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970b3e40-4846-4c0e-ad62-2fae72fb19db" path="/var/lib/kubelet/pods/970b3e40-4846-4c0e-ad62-2fae72fb19db/volumes"
Sep 18 18:46:08.150700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:08.150663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t" event={"ID":"2e428cc9-49f8-49c4-a23d-a9f769b5bb58","Type":"ContainerStarted","Data":"cb71999cac412cfcc84670a3874738a60076cb57293a36e9d8cdc244dac86a94"}
Sep 18 18:46:08.151177 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:08.150780    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:46:08.178957 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:08.178884    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t" podStartSLOduration=2.459343219 podStartE2EDuration="5.17886885s" totalImagesPullingTime="2.719525631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:08.17664389 +0000 UTC m=+411.196440516" watchObservedRunningTime="2026-09-18 18:46:08.17886885 +0000 UTC m=+411.198665476"
Sep 18 18:46:09.104815 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:09.104782    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fshmm"
Sep 18 18:46:09.106682 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:09.106658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8jhwg"
Sep 18 18:46:39.158433 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:46:39.158350    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4687t"
Sep 18 18:47:14.206075 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.206039    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d9vqc"]
Sep 18 18:47:14.209159 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.209141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:14.212046 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.212028    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8gbww\""
Sep 18 18:47:14.212446 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.212432    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 18:47:14.212814 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.212794    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 18:47:14.222604 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.222580    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d9vqc"]
Sep 18 18:47:14.318700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.318647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f475\" (UniqueName: \"kubernetes.io/projected/a7893c22-dbb2-4423-a25c-6e1d8ed72eff-kube-api-access-8f475\") pod \"authorino-operator-76b478bd78-d9vqc\" (UID: \"a7893c22-dbb2-4423-a25c-6e1d8ed72eff\") " pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:14.419776 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.419743    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f475\" (UniqueName: \"kubernetes.io/projected/a7893c22-dbb2-4423-a25c-6e1d8ed72eff-kube-api-access-8f475\") pod \"authorino-operator-76b478bd78-d9vqc\" (UID: \"a7893c22-dbb2-4423-a25c-6e1d8ed72eff\") " pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:14.435209 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.435178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f475\" (UniqueName: \"kubernetes.io/projected/a7893c22-dbb2-4423-a25c-6e1d8ed72eff-kube-api-access-8f475\") pod \"authorino-operator-76b478bd78-d9vqc\" (UID: \"a7893c22-dbb2-4423-a25c-6e1d8ed72eff\") " pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:14.518360 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.518277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:14.799275 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:14.799179    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d9vqc"]
Sep 18 18:47:14.802657 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:47:14.802628    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7893c22_dbb2_4423_a25c_6e1d8ed72eff.slice/crio-37d8f3e81d4a81063f266659e5655863ab1a79db3b22daabdfa0ffa76f17b4e1 WatchSource:0}: Error finding container 37d8f3e81d4a81063f266659e5655863ab1a79db3b22daabdfa0ffa76f17b4e1: Status 404 returned error can't find the container with id 37d8f3e81d4a81063f266659e5655863ab1a79db3b22daabdfa0ffa76f17b4e1
Sep 18 18:47:15.398338 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:15.398301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc" event={"ID":"a7893c22-dbb2-4423-a25c-6e1d8ed72eff","Type":"ContainerStarted","Data":"37d8f3e81d4a81063f266659e5655863ab1a79db3b22daabdfa0ffa76f17b4e1"}
Sep 18 18:47:17.408419 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:17.408385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc" event={"ID":"a7893c22-dbb2-4423-a25c-6e1d8ed72eff","Type":"ContainerStarted","Data":"2d2909173916167760e0febeace91b38a1ef24148585ba978390abb58903ee43"}
Sep 18 18:47:17.408814 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:17.408492    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:17.448818 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:17.448709    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc" podStartSLOduration=1.085299762 podStartE2EDuration="3.448691737s" totalImagesPullingTime="2.363391975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:47:17.44694395 +0000 UTC m=+480.466740575" watchObservedRunningTime="2026-09-18 18:47:17.448691737 +0000 UTC m=+480.468488363"
Sep 18 18:47:28.416504 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:28.416475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-d9vqc"
Sep 18 18:47:38.020950 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.020918    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"]
Sep 18 18:47:38.024359 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.024335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.026503 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.026477    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-99vf4\""
Sep 18 18:47:38.038404 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.038378    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"]
Sep 18 18:47:38.112132 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.112098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdbt\" (UniqueName: \"kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.112293 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.112166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.132116 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.132086    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"]
Sep 18 18:47:38.132351 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:47:38.132332    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-lxdbt], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7" podUID="8382ae60-674b-42de-9e84-ec191b5d969f"
Sep 18 18:47:38.212934 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.212874    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdbt\" (UniqueName: \"kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.213101 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.212963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.213341 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.213325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.224326 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.224295    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdbt\" (UniqueName: \"kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-z6wr7\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.492200 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.492167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.498977 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.498947    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:47:38.616457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.616427    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume\") pod \"8382ae60-674b-42de-9e84-ec191b5d969f\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") "
Sep 18 18:47:38.616457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.616469    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxdbt\" (UniqueName: \"kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt\") pod \"8382ae60-674b-42de-9e84-ec191b5d969f\" (UID: \"8382ae60-674b-42de-9e84-ec191b5d969f\") "
Sep 18 18:47:38.616752 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.616724    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8382ae60-674b-42de-9e84-ec191b5d969f" (UID: "8382ae60-674b-42de-9e84-ec191b5d969f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:47:38.618761 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.618732    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt" (OuterVolumeSpecName: "kube-api-access-lxdbt") pod "8382ae60-674b-42de-9e84-ec191b5d969f" (UID: "8382ae60-674b-42de-9e84-ec191b5d969f"). InnerVolumeSpecName "kube-api-access-lxdbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:47:38.717877 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.717831    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8382ae60-674b-42de-9e84-ec191b5d969f-extensions-socket-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:47:38.717877 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.717866    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxdbt\" (UniqueName: \"kubernetes.io/projected/8382ae60-674b-42de-9e84-ec191b5d969f-kube-api-access-lxdbt\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:47:38.956404 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.956371    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"]
Sep 18 18:47:38.962357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:38.962328    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"]
Sep 18 18:47:39.433565 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:39.433537    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8382ae60-674b-42de-9e84-ec191b5d969f" path="/var/lib/kubelet/pods/8382ae60-674b-42de-9e84-ec191b5d969f/volumes"
Sep 18 18:47:39.495643 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:47:39.495610    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-z6wr7"
Sep 18 18:48:11.451397 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.451313    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"]
Sep 18 18:48:11.454690 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.454668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.457022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.457002    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dn7xk\""
Sep 18 18:48:11.464757 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.464736    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"]
Sep 18 18:48:11.477640 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477764 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477764 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477850 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477769    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477850 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477864    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.477971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.477946    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86g4j\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-kube-api-access-86g4j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.578719 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.578684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.578923 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.578734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.578923 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.578860    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579049 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.578918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579049 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579154 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579154 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579161    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86g4j\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-kube-api-access-86g4j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579223    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579254    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579409 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579381    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.579822 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.579804    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.581395 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.581377    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.581480 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.581412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.593564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.593536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.593746 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.593587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86g4j\" (UniqueName: \"kubernetes.io/projected/711f02f9-dc52-4f11-99a8-a3a5e54738b9-kube-api-access-86g4j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qv5bq\" (UID: \"711f02f9-dc52-4f11-99a8-a3a5e54738b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:11.766280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:11.766184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:12.041099 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:12.041067    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"]
Sep 18 18:48:12.044559 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:48:12.044535    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod711f02f9_dc52_4f11_99a8_a3a5e54738b9.slice/crio-6e4baddaa2118113a48e4e8fd24f0d7ba18c0fbb712b570e45eb8e73c4628ba2 WatchSource:0}: Error finding container 6e4baddaa2118113a48e4e8fd24f0d7ba18c0fbb712b570e45eb8e73c4628ba2: Status 404 returned error can't find the container with id 6e4baddaa2118113a48e4e8fd24f0d7ba18c0fbb712b570e45eb8e73c4628ba2
Sep 18 18:48:12.625624 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:12.625585    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" event={"ID":"711f02f9-dc52-4f11-99a8-a3a5e54738b9","Type":"ContainerStarted","Data":"6e4baddaa2118113a48e4e8fd24f0d7ba18c0fbb712b570e45eb8e73c4628ba2"}
Sep 18 18:48:15.450536 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.450501    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:48:15.450790 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.450580    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:48:15.450790 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.450609    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:48:15.640989 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.640958    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" event={"ID":"711f02f9-dc52-4f11-99a8-a3a5e54738b9","Type":"ContainerStarted","Data":"391291650e8ba165d66ea03c5838c9fc6e664bcfeb22eb912a9ebe739e50ddba"}
Sep 18 18:48:15.662084 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.662025    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podStartSLOduration=1.2582578070000001 podStartE2EDuration="4.662008956s" totalImagesPullingTime="3.403751149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:48:15.659700455 +0000 UTC m=+538.679497083" watchObservedRunningTime="2026-09-18 18:48:15.662008956 +0000 UTC m=+538.681805581"
Sep 18 18:48:15.766610 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:15.766533    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:16.771786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:16.771758    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:17.648984 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:17.648954    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:17.650007 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:17.649988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:48:24.915269 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.915233    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:48:24.918676 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.918655    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:24.920907 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.920868    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 18:48:24.921006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.920980    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bhpf9\""
Sep 18 18:48:24.926971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.926931    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:48:24.956288 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.956255    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:48:24.992235 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.992201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gp8f\" (UniqueName: \"kubernetes.io/projected/b878abc6-1b2d-47dd-a267-984ff431b006-kube-api-access-4gp8f\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:24.992382 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:24.992241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b878abc6-1b2d-47dd-a267-984ff431b006-config-file\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.092688 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.092644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gp8f\" (UniqueName: \"kubernetes.io/projected/b878abc6-1b2d-47dd-a267-984ff431b006-kube-api-access-4gp8f\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.092688 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.092696    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b878abc6-1b2d-47dd-a267-984ff431b006-config-file\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.093330 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.093311    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b878abc6-1b2d-47dd-a267-984ff431b006-config-file\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.101771 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.101746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gp8f\" (UniqueName: \"kubernetes.io/projected/b878abc6-1b2d-47dd-a267-984ff431b006-kube-api-access-4gp8f\") pod \"limitador-limitador-84994f7ffc-fb7v5\" (UID: \"b878abc6-1b2d-47dd-a267-984ff431b006\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.229263 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.229171    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:25.499508 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.499429    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:48:25.502950 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:48:25.502916    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb878abc6_1b2d_47dd_a267_984ff431b006.slice/crio-28d0a42cbea545039e240089ab88588792c01c719aa250327297d17fa5ab443f WatchSource:0}: Error finding container 28d0a42cbea545039e240089ab88588792c01c719aa250327297d17fa5ab443f: Status 404 returned error can't find the container with id 28d0a42cbea545039e240089ab88588792c01c719aa250327297d17fa5ab443f
Sep 18 18:48:25.683698 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:25.683657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5" event={"ID":"b878abc6-1b2d-47dd-a267-984ff431b006","Type":"ContainerStarted","Data":"28d0a42cbea545039e240089ab88588792c01c719aa250327297d17fa5ab443f"}
Sep 18 18:48:26.688857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:26.688764    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5" event={"ID":"b878abc6-1b2d-47dd-a267-984ff431b006","Type":"ContainerStarted","Data":"6d32b7cb6b716ec874e775a7298c39bf151d91b4ea31d520f926197aee40db45"}
Sep 18 18:48:26.689239 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:26.688924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:48:26.704702 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:26.704647    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5" podStartSLOduration=1.811167015 podStartE2EDuration="2.704632051s" totalImagesPullingTime="893.465036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:48:26.703201118 +0000 UTC m=+549.722997769" watchObservedRunningTime="2026-09-18 18:48:26.704632051 +0000 UTC m=+549.724428709"
Sep 18 18:48:37.694803 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:48:37.694770    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"
Sep 18 18:49:39.369822 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:39.369738    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:49:48.558291 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:48.558239    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": read tcp 10.133.0.2:56228->10.133.0.23:15021: read: connection reset by peer" start-of-body=
Sep 18 18:49:48.558651 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:48.558436    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": read tcp 10.133.0.2:56228->10.133.0.23:15021: read: connection reset by peer"
Sep 18 18:49:49.037394 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.037364    2558 generic.go:356] "Generic (PLEG): container finished" podID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerID="391291650e8ba165d66ea03c5838c9fc6e664bcfeb22eb912a9ebe739e50ddba" exitCode=0
Sep 18 18:49:49.037559 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.037442    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" event={"ID":"711f02f9-dc52-4f11-99a8-a3a5e54738b9","Type":"ContainerDied","Data":"391291650e8ba165d66ea03c5838c9fc6e664bcfeb22eb912a9ebe739e50ddba"}
Sep 18 18:49:49.037821 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.037807    2558 scope.go:117] "RemoveContainer" containerID="391291650e8ba165d66ea03c5838c9fc6e664bcfeb22eb912a9ebe739e50ddba"
Sep 18 18:49:49.038786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.038754    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:49:49.038884 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.038838    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:49:49.038884 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:49.038873    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:49:50.043517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:50.043482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" event={"ID":"711f02f9-dc52-4f11-99a8-a3a5e54738b9","Type":"ContainerStarted","Data":"d1731dab957b09148b142a32584f9473f34e5d9806d1254c39773cf369f2001c"}
Sep 18 18:49:50.766943 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:50.766876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:49:50.767118 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:50.767079    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 18 18:49:50.767167 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:50.767138    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 18 18:49:51.766692 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:51.766654    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 18 18:49:51.767122 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:51.766728    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 18 18:49:52.767265 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:52.767225    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 18 18:49:52.767655 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:52.767296    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 18 18:49:53.767318 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:53.767277    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 18 18:49:53.767711 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:53.767346    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 18 18:49:54.767365 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:54.767327    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qv5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 18 18:49:54.767738 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:54.767386    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq" podUID="711f02f9-dc52-4f11-99a8-a3a5e54738b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 18 18:49:55.770012 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:55.769980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:49:55.770373 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:55.770276    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:49:55.771164 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:55.771139    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qv5bq"
Sep 18 18:49:58.604156 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:49:58.604115    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:50:12.199284 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:12.199247    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:50:26.849043 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.849005    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"]
Sep 18 18:50:26.851309 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.851291    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.854449 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.854427    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:50:26.854581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.854504    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:50:26.854641 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.854616    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 18 18:50:26.854699 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.854618    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c6hs5\""
Sep 18 18:50:26.859722 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.859699    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"]
Sep 18 18:50:26.905289 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.905464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905298    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.905464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.905464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905350    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48eed3db-ff21-405b-8ae3-aecda9d00f91-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.905464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905374    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjrvc\" (UniqueName: \"kubernetes.io/projected/48eed3db-ff21-405b-8ae3-aecda9d00f91-kube-api-access-pjrvc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:26.905464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:26.905440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjrvc\" (UniqueName: \"kubernetes.io/projected/48eed3db-ff21-405b-8ae3-aecda9d00f91-kube-api-access-pjrvc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006490    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006539    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.006647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48eed3db-ff21-405b-8ae3-aecda9d00f91-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.007016 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.006985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.007111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.007029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.007111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.007086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.009121 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.009091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48eed3db-ff21-405b-8ae3-aecda9d00f91-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.009414 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.009394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48eed3db-ff21-405b-8ae3-aecda9d00f91-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.014222 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.014201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjrvc\" (UniqueName: \"kubernetes.io/projected/48eed3db-ff21-405b-8ae3-aecda9d00f91-kube-api-access-pjrvc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br\" (UID: \"48eed3db-ff21-405b-8ae3-aecda9d00f91\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.162005 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.161883    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:27.433758 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.433736    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"]
Sep 18 18:50:27.438530 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.438509    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:50:27.803214 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:27.803176    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:50:28.206459 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:28.206364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br" event={"ID":"48eed3db-ff21-405b-8ae3-aecda9d00f91","Type":"ContainerStarted","Data":"4b2d4ff97b577877a99ad021e1130ecf0c0ffdf7aaf44c5bf78d0a5bed81694b"}
Sep 18 18:50:29.212200 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:29.212163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br" event={"ID":"48eed3db-ff21-405b-8ae3-aecda9d00f91","Type":"ContainerStarted","Data":"34fc3a644bac52a51ccffc3e8ef169fb1e6d9424b40157b7888b500a891ead8a"}
Sep 18 18:50:29.230008 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:29.229950    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br" podStartSLOduration=2.260285509 podStartE2EDuration="3.229931763s" totalImagesPullingTime="969.646254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:29.229710848 +0000 UTC m=+672.249507488" watchObservedRunningTime="2026-09-18 18:50:29.229931763 +0000 UTC m=+672.249728376"
Sep 18 18:50:37.162430 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:37.162384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:37.162919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:37.162494    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:37.175263 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:37.175238    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:37.255041 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:37.255013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nt4br"
Sep 18 18:50:37.807811 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:37.807774    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:50:54.543401 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.543366    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"]
Sep 18 18:50:54.547453 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.547431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.549826 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.549794    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 18 18:50:54.555813 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.555789    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"]
Sep 18 18:50:54.557359 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.557477 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.557541 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.557596 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f56850-ffa0-427a-83d1-23c8a979c142-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.557596 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtd6\" (UniqueName: \"kubernetes.io/projected/b8f56850-ffa0-427a-83d1-23c8a979c142-kube-api-access-cjtd6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.557700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.557654    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658340 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f56850-ffa0-427a-83d1-23c8a979c142-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtd6\" (UniqueName: \"kubernetes.io/projected/b8f56850-ffa0-427a-83d1-23c8a979c142-kube-api-access-cjtd6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658416    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658860 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.658860 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.659036 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.658914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.660765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.660737    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f56850-ffa0-427a-83d1-23c8a979c142-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.661126 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.661109    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f56850-ffa0-427a-83d1-23c8a979c142-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.666376 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.666357    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtd6\" (UniqueName: \"kubernetes.io/projected/b8f56850-ffa0-427a-83d1-23c8a979c142-kube-api-access-cjtd6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t\" (UID: \"b8f56850-ffa0-427a-83d1-23c8a979c142\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:54.858940 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:54.858820    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:50:55.130335 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:55.130306    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"]
Sep 18 18:50:55.132742 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:50:55.132707    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f56850_ffa0_427a_83d1_23c8a979c142.slice/crio-0060fd9a538ab78ea74af309af18b1ee75dd2c6138f146abacef32eea3789b7f WatchSource:0}: Error finding container 0060fd9a538ab78ea74af309af18b1ee75dd2c6138f146abacef32eea3789b7f: Status 404 returned error can't find the container with id 0060fd9a538ab78ea74af309af18b1ee75dd2c6138f146abacef32eea3789b7f
Sep 18 18:50:55.322060 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:55.322027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t" event={"ID":"b8f56850-ffa0-427a-83d1-23c8a979c142","Type":"ContainerStarted","Data":"0060fd9a538ab78ea74af309af18b1ee75dd2c6138f146abacef32eea3789b7f"}
Sep 18 18:50:56.110920 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:56.110868    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:50:56.327224 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:56.327195    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t" event={"ID":"b8f56850-ffa0-427a-83d1-23c8a979c142","Type":"ContainerStarted","Data":"868e4c66a02ee4b07256f2ec030b9b80278aab15d63917bd0b18964f91cdf798"}
Sep 18 18:50:56.343604 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:50:56.343541    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t" podStartSLOduration=2.152303776 podStartE2EDuration="2.343520016s" totalImagesPullingTime="191.21624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:56.343118268 +0000 UTC m=+699.362914905" watchObservedRunningTime="2026-09-18 18:50:56.343520016 +0000 UTC m=+699.363316643"
Sep 18 18:51:04.858995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:04.858959    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:51:04.859466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:04.859014    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:51:04.871923 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:04.871866    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:51:05.379984 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:05.379957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnkj7t"
Sep 18 18:51:10.617184 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:10.617100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:51:21.716022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:51:21.715983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:52:52.970489 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:52.970400    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-54446658c9-8fzsh"]
Sep 18 18:52:52.973611 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:52.973589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:52.975828 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:52.975805    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-mdfrk\""
Sep 18 18:52:52.975945 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:52.975807    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Sep 18 18:52:52.981519 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:52.981497    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-54446658c9-8fzsh"]
Sep 18 18:52:53.077581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.077543    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhqd\" (UniqueName: \"kubernetes.io/projected/9de584c2-f80c-43a5-9e84-16bef23811fc-kube-api-access-dqhqd\") pod \"ai-gateway-controller-54446658c9-8fzsh\" (UID: \"9de584c2-f80c-43a5-9e84-16bef23811fc\") " pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:53.178821 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.178784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhqd\" (UniqueName: \"kubernetes.io/projected/9de584c2-f80c-43a5-9e84-16bef23811fc-kube-api-access-dqhqd\") pod \"ai-gateway-controller-54446658c9-8fzsh\" (UID: \"9de584c2-f80c-43a5-9e84-16bef23811fc\") " pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:53.187394 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.187362    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhqd\" (UniqueName: \"kubernetes.io/projected/9de584c2-f80c-43a5-9e84-16bef23811fc-kube-api-access-dqhqd\") pod \"ai-gateway-controller-54446658c9-8fzsh\" (UID: \"9de584c2-f80c-43a5-9e84-16bef23811fc\") " pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:53.284492 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.284453    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:53.551947 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.551925    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-54446658c9-8fzsh"]
Sep 18 18:52:53.557156 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:52:53.557126    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de584c2_f80c_43a5_9e84_16bef23811fc.slice/crio-d0e7c7acf9e6eedf4525c2888e4054a11c29742a34a322fe1a2d017f668c6469 WatchSource:0}: Error finding container d0e7c7acf9e6eedf4525c2888e4054a11c29742a34a322fe1a2d017f668c6469: Status 404 returned error can't find the container with id d0e7c7acf9e6eedf4525c2888e4054a11c29742a34a322fe1a2d017f668c6469
Sep 18 18:52:53.829211 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:53.829127    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh" event={"ID":"9de584c2-f80c-43a5-9e84-16bef23811fc","Type":"ContainerStarted","Data":"d0e7c7acf9e6eedf4525c2888e4054a11c29742a34a322fe1a2d017f668c6469"}
Sep 18 18:52:55.840713 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:55.840675    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh" event={"ID":"9de584c2-f80c-43a5-9e84-16bef23811fc","Type":"ContainerStarted","Data":"6a22903775bfe58cb14ab0ab14377f7f5a5918e32354793b5152bdbb6b4782fd"}
Sep 18 18:52:55.841123 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:55.840809    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:52:55.856006 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:52:55.855943    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh" podStartSLOduration=1.76612177 podStartE2EDuration="3.855924266s" totalImagesPullingTime="2.089802496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:52:55.855823287 +0000 UTC m=+818.875619913" watchObservedRunningTime="2026-09-18 18:52:55.855924266 +0000 UTC m=+818.875720907"
Sep 18 18:53:06.847617 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:06.847586    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-54446658c9-8fzsh"
Sep 18 18:53:25.080139 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.080103    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:53:25.085185 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.085163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.087645 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.087620    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bcb770-openshift-default-dockercfg-lbr7s\""
Sep 18 18:53:25.098511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.098482    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:53:25.257918 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.257852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czzt5\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.257951    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.257971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.257995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.258016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.258059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258106 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.258104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258327 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.258124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.258327 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.258218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359241 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359241 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czzt5\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359264    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359286    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359795 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359795 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.359932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.359858    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.360560 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.360539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.361937 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.361907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.362051 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.361993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.367395 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.367370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czzt5\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.367527 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.367492    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token\") pod \"e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") " pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.396682 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.396652    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:25.671620 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.671528    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:53:25.674417 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:53:25.674394    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03abdc1a_6279_44dd_b273_02bb393e8f25.slice/crio-ddba79f29c41f5cf83590f97c086bb261c93dc36a355d1e141ebf9b1aa50e134 WatchSource:0}: Error finding container ddba79f29c41f5cf83590f97c086bb261c93dc36a355d1e141ebf9b1aa50e134: Status 404 returned error can't find the container with id ddba79f29c41f5cf83590f97c086bb261c93dc36a355d1e141ebf9b1aa50e134
Sep 18 18:53:25.676592 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.676560    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:25.676645 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.676632    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:25.676679 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.676657    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:25.968880 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.968795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerStarted","Data":"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"}
Sep 18 18:53:25.968880 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.968857    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerStarted","Data":"ddba79f29c41f5cf83590f97c086bb261c93dc36a355d1e141ebf9b1aa50e134"}
Sep 18 18:53:25.999590 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:25.999513    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podStartSLOduration=0.999489882 podStartE2EDuration="999.489882ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:25.998706015 +0000 UTC m=+849.018502644" watchObservedRunningTime="2026-09-18 18:53:25.999489882 +0000 UTC m=+849.019286508"
Sep 18 18:53:26.397674 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:26.397620    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:26.402968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:26.402942    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:26.973511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:26.973480    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:26.974513 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:26.974495    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:30.313255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.313225    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 18:53:30.316763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.316737    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.319204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.319162    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:53:30.319422 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.319383    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:53:30.320104 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.320083    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:53:30.320367 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.320323    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-htbnh\""
Sep 18 18:53:30.320468 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.320422    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-9d9f36-serving-cert\""
Sep 18 18:53:30.334254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.334226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 18:53:30.398562 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.398523    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.398751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.398581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn75j\" (UniqueName: \"kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.398751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.398618    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.398751 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.398640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.500026 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.499989    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.500187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.500037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn75j\" (UniqueName: \"kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.500187 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:53:30.500145    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-serving-cert: secret "maas-api-e2e-worker-w3-9d9f36-serving-cert" not found
Sep 18 18:53:30.500187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.500156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.500349 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.500197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.500349 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:53:30.500232    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls podName:b3e2ad0e-fd03-43f6-948b-929fce96c467 nodeName:}" failed. No retries permitted until 2026-09-18 18:53:31.000208563 +0000 UTC m=+854.020005169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls") pod "maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" (UID: "b3e2ad0e-fd03-43f6-948b-929fce96c467") : secret "maas-api-e2e-worker-w3-9d9f36-serving-cert" not found
Sep 18 18:53:30.500850 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.500829    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.502733 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.502714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:30.508204 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:30.508184    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn75j\" (UniqueName: \"kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:31.004670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:31.004632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:31.007512 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:31.007480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:31.229715 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:31.229674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:31.504018 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:31.503988    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 18:53:31.506964 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:53:31.506932    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e2ad0e_fd03_43f6_948b_929fce96c467.slice/crio-327e9eb912d299eb6a3a3eb2f0082007074f6506a33fa9bde75cd2849d827593 WatchSource:0}: Error finding container 327e9eb912d299eb6a3a3eb2f0082007074f6506a33fa9bde75cd2849d827593: Status 404 returned error can't find the container with id 327e9eb912d299eb6a3a3eb2f0082007074f6506a33fa9bde75cd2849d827593
Sep 18 18:53:31.996643 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:31.996604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" event={"ID":"b3e2ad0e-fd03-43f6-948b-929fce96c467","Type":"ContainerStarted","Data":"327e9eb912d299eb6a3a3eb2f0082007074f6506a33fa9bde75cd2849d827593"}
Sep 18 18:53:32.027777 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.027741    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:53:32.032250 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.032226    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.034626 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.034600    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bcb770\""
Sep 18 18:53:32.034735 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.034681    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bcb770-dockercfg-mfdms\""
Sep 18 18:53:32.043335 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.043309    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:53:32.056517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.056488    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:53:32.060491 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.060470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.070032 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.070005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:53:32.112628 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.112590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.112809 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.112690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.112809 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.112734    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4ksb\" (UniqueName: \"kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.112952 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.112836    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt4xd\" (UniqueName: \"kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.214131 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.214090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt4xd\" (UniqueName: \"kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.214295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.214149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.214295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.214217    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.214295 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.214242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4ksb\" (UniqueName: \"kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.214968 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.214946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.215037 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.215017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.223048 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.223010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4ksb\" (UniqueName: \"kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb\") pod \"payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.223181 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.223144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt4xd\" (UniqueName: \"kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd\") pod \"payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.343743 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.343702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:32.372040 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.371995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:32.632023 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.631968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:53:32.634602 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:53:32.634578    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4343e28_58c3_4c1a_ab67_02d9a4b7c4f4.slice/crio-2d404c787fa23a226f360cb0dee252467e58610126b5f6391efc68bcae54da49 WatchSource:0}: Error finding container 2d404c787fa23a226f360cb0dee252467e58610126b5f6391efc68bcae54da49: Status 404 returned error can't find the container with id 2d404c787fa23a226f360cb0dee252467e58610126b5f6391efc68bcae54da49
Sep 18 18:53:32.674881 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:32.674855    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:53:33.002797 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:33.002651    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" event={"ID":"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4","Type":"ContainerStarted","Data":"2d404c787fa23a226f360cb0dee252467e58610126b5f6391efc68bcae54da49"}
Sep 18 18:53:33.080553 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:53:33.080504    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b40a66_ff53_4611_8310_145e6798e36a.slice/crio-b974102819a14b1b2ec1a2e834a10ef63e43c837e277dd5173204dec17e9d30e WatchSource:0}: Error finding container b974102819a14b1b2ec1a2e834a10ef63e43c837e277dd5173204dec17e9d30e: Status 404 returned error can't find the container with id b974102819a14b1b2ec1a2e834a10ef63e43c837e277dd5173204dec17e9d30e
Sep 18 18:53:34.011642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:34.011604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" event={"ID":"b3e2ad0e-fd03-43f6-948b-929fce96c467","Type":"ContainerStarted","Data":"0f02ed344bc139483abbf8f427543b4b44d9044aeddf68423fb469eb9be21cab"}
Sep 18 18:53:34.011642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:34.011675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:34.013473 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:34.013435    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" event={"ID":"44b40a66-ff53-4611-8310-145e6798e36a","Type":"ContainerStarted","Data":"b974102819a14b1b2ec1a2e834a10ef63e43c837e277dd5173204dec17e9d30e"}
Sep 18 18:53:34.072743 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:34.072677    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" podStartSLOduration=2.448315402 podStartE2EDuration="4.072652724s" totalImagesPullingTime="1.624337322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:34.07001376 +0000 UTC m=+857.089810389" watchObservedRunningTime="2026-09-18 18:53:34.072652724 +0000 UTC m=+857.092449350"
Sep 18 18:53:34.332452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:34.332419    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:53:36.027656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.026879    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" event={"ID":"44b40a66-ff53-4611-8310-145e6798e36a","Type":"ContainerStarted","Data":"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"}
Sep 18 18:53:36.027656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.027212    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:36.032660 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.032603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" event={"ID":"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4","Type":"ContainerStarted","Data":"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"}
Sep 18 18:53:36.032660 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.032661    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:36.050513 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.050459    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" podStartSLOduration=1.824704155 podStartE2EDuration="4.050443365s" totalImagesPullingTime="2.22573921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:36.04965028 +0000 UTC m=+859.069446919" watchObservedRunningTime="2026-09-18 18:53:36.050443365 +0000 UTC m=+859.070240004"
Sep 18 18:53:36.075092 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:36.075015    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" podStartSLOduration=1.410219847 podStartE2EDuration="4.074982916s" totalImagesPullingTime="2.664763069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:36.072054055 +0000 UTC m=+859.091850682" watchObservedRunningTime="2026-09-18 18:53:36.074982916 +0000 UTC m=+859.094779545"
Sep 18 18:53:39.057387 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.057360    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:53:39.060698 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.060683    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.063511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.063474    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6cc65f16\""
Sep 18 18:53:39.063684 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.063665    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6cc65f16-dockercfg-dl86c\""
Sep 18 18:53:39.076569 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.076537    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:53:39.177867 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.177833    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf844\" (UniqueName: \"kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.178058 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.177945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.278734 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.278693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf844\" (UniqueName: \"kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.278930 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.278826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.279594 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.279573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.286862 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.286828    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf844\" (UniqueName: \"kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844\") pod \"payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") " pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.371051 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.370961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:39.667932 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:39.667827    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:53:39.671275 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:53:39.671246    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eda5d7b_f4ca_45a5_b64c_366dd27f6d1e.slice/crio-78959ff9397f14207435e86ec8f2b4e63fb1fb019c04171c5cc76ba485e90415 WatchSource:0}: Error finding container 78959ff9397f14207435e86ec8f2b4e63fb1fb019c04171c5cc76ba485e90415: Status 404 returned error can't find the container with id 78959ff9397f14207435e86ec8f2b4e63fb1fb019c04171c5cc76ba485e90415
Sep 18 18:53:40.025833 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:40.025807    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 18:53:40.054109 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:40.054068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" event={"ID":"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e","Type":"ContainerStarted","Data":"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"}
Sep 18 18:53:40.054267 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:40.054132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" event={"ID":"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e","Type":"ContainerStarted","Data":"78959ff9397f14207435e86ec8f2b4e63fb1fb019c04171c5cc76ba485e90415"}
Sep 18 18:53:40.054319 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:40.054299    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:40.071939 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:40.071862    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" podStartSLOduration=1.071845552 podStartE2EDuration="1.071845552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:40.070236989 +0000 UTC m=+863.090033617" watchObservedRunningTime="2026-09-18 18:53:40.071845552 +0000 UTC m=+863.091642225"
Sep 18 18:53:43.016071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:43.016033    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:53:43.016434 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:43.016102    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:53:45.080201 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.080167    2558 generic.go:356] "Generic (PLEG): container finished" podID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerID="36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec" exitCode=0
Sep 18 18:53:45.080708 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.080229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerDied","Data":"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"}
Sep 18 18:53:45.080708 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.080694    2558 scope.go:117] "RemoveContainer" containerID="36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"
Sep 18 18:53:45.081845 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.081815    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:45.081922 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.081902    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:45.081975 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:45.081943    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:53:46.086601 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:46.086567    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerStarted","Data":"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"}
Sep 18 18:53:46.399002 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:46.397281    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:46.399002 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:46.397450    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Sep 18 18:53:46.399002 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:46.397511    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Sep 18 18:53:47.038790 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:47.038765    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:53:47.039113 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:47.039088    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:53:47.398229 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:47.398127    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Sep 18 18:53:47.398805 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:47.398768    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Sep 18 18:53:48.397450 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:48.397413    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Sep 18 18:53:48.397759 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:48.397727    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Sep 18 18:53:49.397973 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:49.397927    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Sep 18 18:53:49.398549 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:49.398517    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Sep 18 18:53:50.398034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:50.397994    2558 patch_prober.go:28] interesting pod/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Sep 18 18:53:50.398647 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:50.398617    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Sep 18 18:53:51.061537 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:51.061506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:53:51.401372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:51.401276    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:51.401754 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:51.401555    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:51.402347 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:51.402322    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:53:52.042619 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:53:52.042581    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:54:05.909555 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:05.909520    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:54:05.917253 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:05.917230    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:05.919780 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:05.919757    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-897f61-serving-cert\""
Sep 18 18:54:05.924916 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:05.924864    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:54:06.029398 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.029347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.029586 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.029456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7jgx\" (UniqueName: \"kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.029586 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.029504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.029586 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.029550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.130538 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.130503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7jgx\" (UniqueName: \"kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.130538 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.130553    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.130795 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.130597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.130795 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.130682    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.131268 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.131241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.133261 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.133239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.133357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.133312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.140529 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.140503    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7jgx\" (UniqueName: \"kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx\") pod \"maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.247947 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.247837    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:06.522249 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.522226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:54:06.524218 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:06.524190    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c54852_7b7c_430f_994b_bd1490d243db.slice/crio-177ae93c16f30d5c2d2b32b54208a306b42d7125bbe9ab65a9e195deffb9f937 WatchSource:0}: Error finding container 177ae93c16f30d5c2d2b32b54208a306b42d7125bbe9ab65a9e195deffb9f937: Status 404 returned error can't find the container with id 177ae93c16f30d5c2d2b32b54208a306b42d7125bbe9ab65a9e195deffb9f937
Sep 18 18:54:06.557028 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.556997    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:54:06.561392 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.561376    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.563931 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.563912    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-897f61\""
Sep 18 18:54:06.564011 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.563948    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-897f61-dockercfg-xtcfz\""
Sep 18 18:54:06.571008 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.570982    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:54:06.737168 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.737126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.737168 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.737167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7b5t\" (UniqueName: \"kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.838078 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.838046    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.838078 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.838084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7b5t\" (UniqueName: \"kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.838736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.838715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.847487 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.847464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7b5t\" (UniqueName: \"kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t\") pod \"payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:06.872331 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:06.872306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:07.162310 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.162284    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:54:07.164630 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:07.164601    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830a2656_fe5b_44f4_93ba_dd3d7c6eb5e0.slice/crio-da09cd8da1e84d1cd294df2153d8eb65498000d6c0a8d8d88419d80b41b59155 WatchSource:0}: Error finding container da09cd8da1e84d1cd294df2153d8eb65498000d6c0a8d8d88419d80b41b59155: Status 404 returned error can't find the container with id da09cd8da1e84d1cd294df2153d8eb65498000d6c0a8d8d88419d80b41b59155
Sep 18 18:54:07.204307 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.204274    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" event={"ID":"b7c54852-7b7c-430f-994b-bd1490d243db","Type":"ContainerStarted","Data":"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"}
Sep 18 18:54:07.204436 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.204341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:07.204436 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.204365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" event={"ID":"b7c54852-7b7c-430f-994b-bd1490d243db","Type":"ContainerStarted","Data":"177ae93c16f30d5c2d2b32b54208a306b42d7125bbe9ab65a9e195deffb9f937"}
Sep 18 18:54:07.205618 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.205592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" event={"ID":"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0","Type":"ContainerStarted","Data":"da09cd8da1e84d1cd294df2153d8eb65498000d6c0a8d8d88419d80b41b59155"}
Sep 18 18:54:07.225825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.225772    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" podStartSLOduration=1.89409984 podStartE2EDuration="2.225751206s" totalImagesPullingTime="331.651366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:07.223947687 +0000 UTC m=+890.243744307" watchObservedRunningTime="2026-09-18 18:54:07.225751206 +0000 UTC m=+890.245547832"
Sep 18 18:54:07.518804 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.518566    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:54:07.518972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.518880    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" podUID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" containerName="payload-processing" containerID="cri-o://acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34" gracePeriod=30
Sep 18 18:54:07.777357 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.777330    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:54:07.952518 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.952472    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume\") pod \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") "
Sep 18 18:54:07.952700 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.952525    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf844\" (UniqueName: \"kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844\") pod \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\" (UID: \"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e\") "
Sep 18 18:54:07.952859 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.952831    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" (UID: "0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:07.954800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:07.954775    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844" (OuterVolumeSpecName: "kube-api-access-sf844") pod "0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" (UID: "0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e"). InnerVolumeSpecName "kube-api-access-sf844". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:08.053285 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.053200    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:08.053285 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.053229    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf844\" (UniqueName: \"kubernetes.io/projected/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e-kube-api-access-sf844\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:08.213374 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.213337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" event={"ID":"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0","Type":"ContainerStarted","Data":"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"}
Sep 18 18:54:08.213760 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.213513    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:08.214650 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.214625    2558 generic.go:356] "Generic (PLEG): container finished" podID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" containerID="acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34" exitCode=0
Sep 18 18:54:08.214775 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.214660    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"
Sep 18 18:54:08.214775 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.214699    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" event={"ID":"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e","Type":"ContainerDied","Data":"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"}
Sep 18 18:54:08.214775 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.214741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z" event={"ID":"0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e","Type":"ContainerDied","Data":"78959ff9397f14207435e86ec8f2b4e63fb1fb019c04171c5cc76ba485e90415"}
Sep 18 18:54:08.214775 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.214768    2558 scope.go:117] "RemoveContainer" containerID="acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"
Sep 18 18:54:08.223585 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.223570    2558 scope.go:117] "RemoveContainer" containerID="acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"
Sep 18 18:54:08.223837 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:54:08.223818    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34\": container with ID starting with acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34 not found: ID does not exist" containerID="acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"
Sep 18 18:54:08.223875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.223845    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34"} err="failed to get container status \"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34\": rpc error: code = NotFound desc = could not find container \"acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34\": container with ID starting with acd337075ca805818348e1d6a70413c638f383f68e3ae4ad0552da6ac7076f34 not found: ID does not exist"
Sep 18 18:54:08.252407 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.252362    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" podStartSLOduration=2.252348077 podStartE2EDuration="2.252348077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:08.249635425 +0000 UTC m=+891.269432049" watchObservedRunningTime="2026-09-18 18:54:08.252348077 +0000 UTC m=+891.272144702"
Sep 18 18:54:08.286102 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.286069    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:54:08.296575 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:08.296539    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6cc65f16-95586fdc5-dnv7z"]
Sep 18 18:54:09.433533 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:09.433505    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" path="/var/lib/kubelet/pods/0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e/volumes"
Sep 18 18:54:12.612099 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:12.612060    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:54:13.221158 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.221131    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:54:13.456370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.456342    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:13.456760 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.456742    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" containerName="payload-processing"
Sep 18 18:54:13.456845 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.456763    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" containerName="payload-processing"
Sep 18 18:54:13.456922 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.456881    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eda5d7b-f4ca-45a5-b64c-366dd27f6d1e" containerName="payload-processing"
Sep 18 18:54:13.459664 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.459644    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.466340 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.466310    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e8c645ee\""
Sep 18 18:54:13.471348 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.471290    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:13.478825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.478802    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:13.482569 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.482552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.484728 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.484712    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e8c645ee-dockercfg-g8chk\""
Sep 18 18:54:13.495358 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.495334    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:13.604592 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.604557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.604882 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.604855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g64fl\" (UniqueName: \"kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.605099 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.605081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfbsp\" (UniqueName: \"kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.605267 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.605251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.706814 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.706768    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.707270 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.706840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g64fl\" (UniqueName: \"kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.707270 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.706884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfbsp\" (UniqueName: \"kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.707270 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.706981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.707669 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.707646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.707738 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.707654    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.715122 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.715092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g64fl\" (UniqueName: \"kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl\") pod \"payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.715312 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.715290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfbsp\" (UniqueName: \"kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp\") pod \"payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") " pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:13.778266 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.778239    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:13.795307 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:13.795280    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:14.056925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.056871    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:14.058739 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:14.058713    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc7b9c0_fde1_4231_8b4d_e845d5c052b2.slice/crio-3a0cb5c6895eb9e8bd16cf07f44eb5ecf1e2e80432fd3bdef905d9cd431ece32 WatchSource:0}: Error finding container 3a0cb5c6895eb9e8bd16cf07f44eb5ecf1e2e80432fd3bdef905d9cd431ece32: Status 404 returned error can't find the container with id 3a0cb5c6895eb9e8bd16cf07f44eb5ecf1e2e80432fd3bdef905d9cd431ece32
Sep 18 18:54:14.086914 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.086859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:14.091072 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:14.091048    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d14fd9f_094d_436a_a69b_bf9e990108a4.slice/crio-4e9a6f1cf9869204379ecc49dfd583e00bdad14916c9b1483a15070d5ea9b4f4 WatchSource:0}: Error finding container 4e9a6f1cf9869204379ecc49dfd583e00bdad14916c9b1483a15070d5ea9b4f4: Status 404 returned error can't find the container with id 4e9a6f1cf9869204379ecc49dfd583e00bdad14916c9b1483a15070d5ea9b4f4
Sep 18 18:54:14.246474 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.246434    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" event={"ID":"1d14fd9f-094d-436a-a69b-bf9e990108a4","Type":"ContainerStarted","Data":"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"}
Sep 18 18:54:14.246655 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.246616    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:14.246712 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.246652    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" event={"ID":"1d14fd9f-094d-436a-a69b-bf9e990108a4","Type":"ContainerStarted","Data":"4e9a6f1cf9869204379ecc49dfd583e00bdad14916c9b1483a15070d5ea9b4f4"}
Sep 18 18:54:14.248083 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.248058    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" event={"ID":"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2","Type":"ContainerStarted","Data":"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"}
Sep 18 18:54:14.248172 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.248100    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" event={"ID":"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2","Type":"ContainerStarted","Data":"3a0cb5c6895eb9e8bd16cf07f44eb5ecf1e2e80432fd3bdef905d9cd431ece32"}
Sep 18 18:54:14.248219 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.248198    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:14.264849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.264799    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" podStartSLOduration=1.2647795450000001 podStartE2EDuration="1.264779545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:14.264070597 +0000 UTC m=+897.283867224" watchObservedRunningTime="2026-09-18 18:54:14.264779545 +0000 UTC m=+897.284576172"
Sep 18 18:54:14.281031 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:14.280985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" podStartSLOduration=1.280967494 podStartE2EDuration="1.280967494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:14.279960396 +0000 UTC m=+897.299757022" watchObservedRunningTime="2026-09-18 18:54:14.280967494 +0000 UTC m=+897.300764121"
Sep 18 18:54:19.222509 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:19.222474    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:54:25.255786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:25.255739    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:25.256261 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:25.255833    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:33.519362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:33.519322    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:54:41.272299 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.272264    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:41.272773 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.272525    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" podUID="1d14fd9f-094d-436a-a69b-bf9e990108a4" containerName="payload-processing" containerID="cri-o://6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0" gracePeriod=30
Sep 18 18:54:41.282396 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.282366    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:41.282653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.282622    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" podUID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" containerName="payload-pre-processing" containerID="cri-o://ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d" gracePeriod=30
Sep 18 18:54:41.535526 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.535501    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:41.572505 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.572455    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:41.662305 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume\") pod \"1d14fd9f-094d-436a-a69b-bf9e990108a4\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") "
Sep 18 18:54:41.662496 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662319    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume\") pod \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") "
Sep 18 18:54:41.662496 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662390    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g64fl\" (UniqueName: \"kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl\") pod \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\" (UID: \"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2\") "
Sep 18 18:54:41.662496 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662453    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfbsp\" (UniqueName: \"kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp\") pod \"1d14fd9f-094d-436a-a69b-bf9e990108a4\" (UID: \"1d14fd9f-094d-436a-a69b-bf9e990108a4\") "
Sep 18 18:54:41.662734 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662709    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d14fd9f-094d-436a-a69b-bf9e990108a4" (UID: "1d14fd9f-094d-436a-a69b-bf9e990108a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:41.662814 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.662783    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" (UID: "2bc7b9c0-fde1-4231-8b4d-e845d5c052b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:41.664736 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.664699    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl" (OuterVolumeSpecName: "kube-api-access-g64fl") pod "2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" (UID: "2bc7b9c0-fde1-4231-8b4d-e845d5c052b2"). InnerVolumeSpecName "kube-api-access-g64fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:41.664835 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.664734    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp" (OuterVolumeSpecName: "kube-api-access-tfbsp") pod "1d14fd9f-094d-436a-a69b-bf9e990108a4" (UID: "1d14fd9f-094d-436a-a69b-bf9e990108a4"). InnerVolumeSpecName "kube-api-access-tfbsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:41.763328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.763291    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g64fl\" (UniqueName: \"kubernetes.io/projected/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-kube-api-access-g64fl\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.763328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.763324    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfbsp\" (UniqueName: \"kubernetes.io/projected/1d14fd9f-094d-436a-a69b-bf9e990108a4-kube-api-access-tfbsp\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.763328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.763333    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d14fd9f-094d-436a-a69b-bf9e990108a4-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.763564 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:41.763343    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:54:42.382312 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.382281    2558 generic.go:356] "Generic (PLEG): container finished" podID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" containerID="ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d" exitCode=0
Sep 18 18:54:42.382737 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.382328    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"
Sep 18 18:54:42.382737 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.382372    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" event={"ID":"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2","Type":"ContainerDied","Data":"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"}
Sep 18 18:54:42.382737 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.382438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8" event={"ID":"2bc7b9c0-fde1-4231-8b4d-e845d5c052b2","Type":"ContainerDied","Data":"3a0cb5c6895eb9e8bd16cf07f44eb5ecf1e2e80432fd3bdef905d9cd431ece32"}
Sep 18 18:54:42.382737 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.382469    2558 scope.go:117] "RemoveContainer" containerID="ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"
Sep 18 18:54:42.383865 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.383843    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d14fd9f-094d-436a-a69b-bf9e990108a4" containerID="6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0" exitCode=0
Sep 18 18:54:42.383995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.383884    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"
Sep 18 18:54:42.383995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.383924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" event={"ID":"1d14fd9f-094d-436a-a69b-bf9e990108a4","Type":"ContainerDied","Data":"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"}
Sep 18 18:54:42.383995 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.383990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc" event={"ID":"1d14fd9f-094d-436a-a69b-bf9e990108a4","Type":"ContainerDied","Data":"4e9a6f1cf9869204379ecc49dfd583e00bdad14916c9b1483a15070d5ea9b4f4"}
Sep 18 18:54:42.393882 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.393801    2558 scope.go:117] "RemoveContainer" containerID="ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"
Sep 18 18:54:42.394596 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:54:42.394220    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d\": container with ID starting with ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d not found: ID does not exist" containerID="ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"
Sep 18 18:54:42.394596 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.394251    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d"} err="failed to get container status \"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d\": rpc error: code = NotFound desc = could not find container \"ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d\": container with ID starting with ffd0f1c1e899c2e552a2b32e89c971d477446d68723d51fdcaa7eabbe84ee78d not found: ID does not exist"
Sep 18 18:54:42.394596 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.394273    2558 scope.go:117] "RemoveContainer" containerID="6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"
Sep 18 18:54:42.403617 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.403600    2558 scope.go:117] "RemoveContainer" containerID="6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"
Sep 18 18:54:42.403989 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:54:42.403884    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0\": container with ID starting with 6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0 not found: ID does not exist" containerID="6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"
Sep 18 18:54:42.404089 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.403999    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0"} err="failed to get container status \"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0\": rpc error: code = NotFound desc = could not find container \"6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0\": container with ID starting with 6d6e6527bb2729ea2694e42247074c561291fa74038ebfece89ccf45978532c0 not found: ID does not exist"
Sep 18 18:54:42.416907 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.416858    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:42.425752 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.425725    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e8c645ee-64876bcfb6-4qvr8"]
Sep 18 18:54:42.436013 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.435989    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:42.438530 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:42.438510    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e8c645ee-b86775758-nm5rc"]
Sep 18 18:54:43.434507 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:43.434473    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d14fd9f-094d-436a-a69b-bf9e990108a4" path="/var/lib/kubelet/pods/1d14fd9f-094d-436a-a69b-bf9e990108a4/volumes"
Sep 18 18:54:43.434871 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:43.434824    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" path="/var/lib/kubelet/pods/2bc7b9c0-fde1-4231-8b4d-e845d5c052b2/volumes"
Sep 18 18:54:48.511392 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:48.511350    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:54:49.598508 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598474    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:54:49.598925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598832    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" containerName="payload-pre-processing"
Sep 18 18:54:49.598925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598843    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" containerName="payload-pre-processing"
Sep 18 18:54:49.598925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598853    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d14fd9f-094d-436a-a69b-bf9e990108a4" containerName="payload-processing"
Sep 18 18:54:49.598925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598859    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d14fd9f-094d-436a-a69b-bf9e990108a4" containerName="payload-processing"
Sep 18 18:54:49.598925 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598921    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc7b9c0-fde1-4231-8b4d-e845d5c052b2" containerName="payload-pre-processing"
Sep 18 18:54:49.599084 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.598934    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d14fd9f-094d-436a-a69b-bf9e990108a4" containerName="payload-processing"
Sep 18 18:54:49.603066 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.603046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.605414 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.605391    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b0eada48\""
Sep 18 18:54:49.611388 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.611364    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:54:49.643473 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.643438    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:54:49.646706 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.646690    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.651964 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.651941    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b0eada48-dockercfg-lcn45\""
Sep 18 18:54:49.657254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.657230    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:54:49.738257 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.738222    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfpnc\" (UniqueName: \"kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.738431 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.738263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.738431 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.738395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw72t\" (UniqueName: \"kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.738431 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.738418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.838863 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.838820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfpnc\" (UniqueName: \"kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.838863 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.838865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.839173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.838965    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw72t\" (UniqueName: \"kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.839173 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.838987    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.839629 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.839610    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.839726 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.839707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.847327 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.847302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfpnc\" (UniqueName: \"kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc\") pod \"payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.847621 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.847605    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw72t\" (UniqueName: \"kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t\") pod \"payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") " pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:49.914289 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.914201    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:49.958126 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:49.958084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:50.186787 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.186760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:54:50.188370 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:50.188339    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1db39b1_5db2_400a_928f_ca1102343a39.slice/crio-a55cf383bdcd9d1c551890a54eb271b9e9762685e778b824c741c60499d64a8a WatchSource:0}: Error finding container a55cf383bdcd9d1c551890a54eb271b9e9762685e778b824c741c60499d64a8a: Status 404 returned error can't find the container with id a55cf383bdcd9d1c551890a54eb271b9e9762685e778b824c741c60499d64a8a
Sep 18 18:54:50.241906 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.241860    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:54:50.243925 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:54:50.243871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca14061_fa9c_43ae_a21b_7e2e3a30b634.slice/crio-ff706290f672c100ca4d38ffc7cfc7613bd7c4e112c9293ba0572b87e281c04e WatchSource:0}: Error finding container ff706290f672c100ca4d38ffc7cfc7613bd7c4e112c9293ba0572b87e281c04e: Status 404 returned error can't find the container with id ff706290f672c100ca4d38ffc7cfc7613bd7c4e112c9293ba0572b87e281c04e
Sep 18 18:54:50.424884 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.424850    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" event={"ID":"2ca14061-fa9c-43ae-a21b-7e2e3a30b634","Type":"ContainerStarted","Data":"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"}
Sep 18 18:54:50.425065 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.424934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" event={"ID":"2ca14061-fa9c-43ae-a21b-7e2e3a30b634","Type":"ContainerStarted","Data":"ff706290f672c100ca4d38ffc7cfc7613bd7c4e112c9293ba0572b87e281c04e"}
Sep 18 18:54:50.425065 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.424966    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:54:50.426734 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.426705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" event={"ID":"c1db39b1-5db2-400a-928f-ca1102343a39","Type":"ContainerStarted","Data":"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"}
Sep 18 18:54:50.426868 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.426759    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" event={"ID":"c1db39b1-5db2-400a-928f-ca1102343a39","Type":"ContainerStarted","Data":"a55cf383bdcd9d1c551890a54eb271b9e9762685e778b824c741c60499d64a8a"}
Sep 18 18:54:50.426868 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.426808    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:54:50.441628 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.441478    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" podStartSLOduration=1.441459137 podStartE2EDuration="1.441459137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:50.440072912 +0000 UTC m=+933.459869537" watchObservedRunningTime="2026-09-18 18:54:50.441459137 +0000 UTC m=+933.461255763"
Sep 18 18:54:50.456716 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:54:50.456663    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" podStartSLOduration=1.456645449 podStartE2EDuration="1.456645449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:50.454600884 +0000 UTC m=+933.474397514" watchObservedRunningTime="2026-09-18 18:54:50.456645449 +0000 UTC m=+933.476442074"
Sep 18 18:55:01.436473 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:01.436445    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:55:01.436866 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:01.436514    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:55:07.308946 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:07.308910    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:55:24.702306 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:24.702264    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:55:27.586983 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.586946    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:55:27.587382 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.587209    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" podUID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" containerName="payload-processing" containerID="cri-o://260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a" gracePeriod=30
Sep 18 18:55:27.594269 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.594060    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:55:27.594413 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.594311    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" podUID="c1db39b1-5db2-400a-928f-ca1102343a39" containerName="payload-pre-processing" containerID="cri-o://6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c" gracePeriod=30
Sep 18 18:55:27.865090 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.865065    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:55:27.867753 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.867731    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:55:27.978384 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978344    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume\") pod \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") "
Sep 18 18:55:27.978561 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978453    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfpnc\" (UniqueName: \"kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc\") pod \"c1db39b1-5db2-400a-928f-ca1102343a39\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") "
Sep 18 18:55:27.978561 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978478    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume\") pod \"c1db39b1-5db2-400a-928f-ca1102343a39\" (UID: \"c1db39b1-5db2-400a-928f-ca1102343a39\") "
Sep 18 18:55:27.978561 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978554    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw72t\" (UniqueName: \"kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t\") pod \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\" (UID: \"2ca14061-fa9c-43ae-a21b-7e2e3a30b634\") "
Sep 18 18:55:27.978774 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978742    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2ca14061-fa9c-43ae-a21b-7e2e3a30b634" (UID: "2ca14061-fa9c-43ae-a21b-7e2e3a30b634"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:55:27.978854 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.978834    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:55:27.979267 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.979235    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1db39b1-5db2-400a-928f-ca1102343a39" (UID: "c1db39b1-5db2-400a-928f-ca1102343a39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:55:27.981737 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.981712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t" (OuterVolumeSpecName: "kube-api-access-dw72t") pod "2ca14061-fa9c-43ae-a21b-7e2e3a30b634" (UID: "2ca14061-fa9c-43ae-a21b-7e2e3a30b634"). InnerVolumeSpecName "kube-api-access-dw72t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:55:27.981832 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:27.981734    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc" (OuterVolumeSpecName: "kube-api-access-zfpnc") pod "c1db39b1-5db2-400a-928f-ca1102343a39" (UID: "c1db39b1-5db2-400a-928f-ca1102343a39"). InnerVolumeSpecName "kube-api-access-zfpnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:55:28.079629 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.079596    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw72t\" (UniqueName: \"kubernetes.io/projected/2ca14061-fa9c-43ae-a21b-7e2e3a30b634-kube-api-access-dw72t\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:55:28.079629 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.079625    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfpnc\" (UniqueName: \"kubernetes.io/projected/c1db39b1-5db2-400a-928f-ca1102343a39-kube-api-access-zfpnc\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:55:28.079629 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.079636    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1db39b1-5db2-400a-928f-ca1102343a39-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:55:28.613490 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.613460    2558 generic.go:356] "Generic (PLEG): container finished" podID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" containerID="260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a" exitCode=0
Sep 18 18:55:28.613875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.613508    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"
Sep 18 18:55:28.613875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.613534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" event={"ID":"2ca14061-fa9c-43ae-a21b-7e2e3a30b634","Type":"ContainerDied","Data":"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"}
Sep 18 18:55:28.613875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.613594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h" event={"ID":"2ca14061-fa9c-43ae-a21b-7e2e3a30b634","Type":"ContainerDied","Data":"ff706290f672c100ca4d38ffc7cfc7613bd7c4e112c9293ba0572b87e281c04e"}
Sep 18 18:55:28.613875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.613631    2558 scope.go:117] "RemoveContainer" containerID="260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"
Sep 18 18:55:28.615034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.614870    2558 generic.go:356] "Generic (PLEG): container finished" podID="c1db39b1-5db2-400a-928f-ca1102343a39" containerID="6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c" exitCode=0
Sep 18 18:55:28.615034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.614975    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"
Sep 18 18:55:28.615034 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.615016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" event={"ID":"c1db39b1-5db2-400a-928f-ca1102343a39","Type":"ContainerDied","Data":"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"}
Sep 18 18:55:28.615161 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.615069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj" event={"ID":"c1db39b1-5db2-400a-928f-ca1102343a39","Type":"ContainerDied","Data":"a55cf383bdcd9d1c551890a54eb271b9e9762685e778b824c741c60499d64a8a"}
Sep 18 18:55:28.624188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.624172    2558 scope.go:117] "RemoveContainer" containerID="260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"
Sep 18 18:55:28.624552 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:55:28.624529    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a\": container with ID starting with 260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a not found: ID does not exist" containerID="260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"
Sep 18 18:55:28.624655 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.624558    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a"} err="failed to get container status \"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a\": rpc error: code = NotFound desc = could not find container \"260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a\": container with ID starting with 260ab81975bff80362d265b56f24d10d5d9244747030960f8260f1fbe48ced7a not found: ID does not exist"
Sep 18 18:55:28.624655 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.624576    2558 scope.go:117] "RemoveContainer" containerID="6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"
Sep 18 18:55:28.632642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.632614    2558 scope.go:117] "RemoveContainer" containerID="6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"
Sep 18 18:55:28.632933 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:55:28.632907    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c\": container with ID starting with 6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c not found: ID does not exist" containerID="6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"
Sep 18 18:55:28.633021 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.632944    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c"} err="failed to get container status \"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c\": rpc error: code = NotFound desc = could not find container \"6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c\": container with ID starting with 6a91aedcc158482571139b2dfbac8725df0b7f15408d04cf21a754affe88a86c not found: ID does not exist"
Sep 18 18:55:28.636586 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.636562    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:55:28.640469 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.640448    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0eada48-57f5d65cbf-kmg8h"]
Sep 18 18:55:28.661977 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.661951    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:55:28.672182 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:28.672156    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0eada48-67bc5d8bd6-mk4pj"]
Sep 18 18:55:29.434957 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.434925    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" path="/var/lib/kubelet/pods/2ca14061-fa9c-43ae-a21b-7e2e3a30b634/volumes"
Sep 18 18:55:29.435324 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.435308    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1db39b1-5db2-400a-928f-ca1102343a39" path="/var/lib/kubelet/pods/c1db39b1-5db2-400a-928f-ca1102343a39/volumes"
Sep 18 18:55:29.924549 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.924512    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:55:29.925058 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925039    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1db39b1-5db2-400a-928f-ca1102343a39" containerName="payload-pre-processing"
Sep 18 18:55:29.925132 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925062    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1db39b1-5db2-400a-928f-ca1102343a39" containerName="payload-pre-processing"
Sep 18 18:55:29.925132 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925092    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" containerName="payload-processing"
Sep 18 18:55:29.925132 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925101    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" containerName="payload-processing"
Sep 18 18:55:29.925276 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925198    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1db39b1-5db2-400a-928f-ca1102343a39" containerName="payload-pre-processing"
Sep 18 18:55:29.925276 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.925211    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca14061-fa9c-43ae-a21b-7e2e3a30b634" containerName="payload-processing"
Sep 18 18:55:29.929421 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.929398    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:29.931875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.931858    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d2dcf91d-openshift-default-dockercfg-2n4hh\""
Sep 18 18:55:29.947849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:29.947824    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:55:30.097967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.097930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.097967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.097968    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098175 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098175 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098175 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098175 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098302 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098185    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098302 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbn9c\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.098302 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.098271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199832    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbn9c\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.199926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200177 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200177 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.199964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200177 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.200168    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200328 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.200204    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200493 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.200469    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200570 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.200478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.200835 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.200790    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.202451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.202426    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.202635 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.202618    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.213472 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.213444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbn9c\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.215588 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.215570    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token\") pod \"e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") " pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.239694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.239663    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:30.516228 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.516104    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:55:30.518482 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:55:30.518454    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd643b5e2_d0ba_4b67_ab88_fa97e2ccc912.slice/crio-107ffa65636475cd26f035e2dc59f9ae86ccede795df2aecb6acc2a51ed2074d WatchSource:0}: Error finding container 107ffa65636475cd26f035e2dc59f9ae86ccede795df2aecb6acc2a51ed2074d: Status 404 returned error can't find the container with id 107ffa65636475cd26f035e2dc59f9ae86ccede795df2aecb6acc2a51ed2074d
Sep 18 18:55:30.520604 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.520574    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:30.520654 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.520639    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:30.520702 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.520666    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:30.628935 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.628867    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerStarted","Data":"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"}
Sep 18 18:55:30.629122 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.628942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerStarted","Data":"107ffa65636475cd26f035e2dc59f9ae86ccede795df2aecb6acc2a51ed2074d"}
Sep 18 18:55:30.649953 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:30.649866    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podStartSLOduration=1.6498512459999999 podStartE2EDuration="1.649851246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:30.648191997 +0000 UTC m=+973.667988637" watchObservedRunningTime="2026-09-18 18:55:30.649851246 +0000 UTC m=+973.669647874"
Sep 18 18:55:31.240021 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:31.239989    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:31.245024 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:31.244998    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:31.633566 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:31.633536    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:31.634657 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:31.634629    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:34.184476 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.184436    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:55:34.188269 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.188244    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.191099 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.191075    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d2dcf91d\""
Sep 18 18:55:34.202809 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.202779    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:55:34.337119 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.337072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.337298 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.337196    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w9mb\" (UniqueName: \"kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.438510 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.438401    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.438683 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.438549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w9mb\" (UniqueName: \"kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.439197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.439175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.448656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.448628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w9mb\" (UniqueName: \"kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb\") pod \"payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") " pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.500623 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.500590    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:34.776582 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:34.776556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:55:34.778404 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:55:34.778382    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a266793_80d3_4cc3_b38d_6811090f8fa8.slice/crio-444751ff7a411c8840e9843acb3df2cb107690dd870bc9ea9cd05c35fe3c8451 WatchSource:0}: Error finding container 444751ff7a411c8840e9843acb3df2cb107690dd870bc9ea9cd05c35fe3c8451: Status 404 returned error can't find the container with id 444751ff7a411c8840e9843acb3df2cb107690dd870bc9ea9cd05c35fe3c8451
Sep 18 18:55:35.655285 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:35.655242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" event={"ID":"3a266793-80d3-4cc3-b38d-6811090f8fa8","Type":"ContainerStarted","Data":"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"}
Sep 18 18:55:35.655758 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:35.655316    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" event={"ID":"3a266793-80d3-4cc3-b38d-6811090f8fa8","Type":"ContainerStarted","Data":"444751ff7a411c8840e9843acb3df2cb107690dd870bc9ea9cd05c35fe3c8451"}
Sep 18 18:55:35.655971 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:35.655943    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:35.673613 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:35.673567    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" podStartSLOduration=1.673551434 podStartE2EDuration="1.673551434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:35.672959408 +0000 UTC m=+978.692756035" watchObservedRunningTime="2026-09-18 18:55:35.673551434 +0000 UTC m=+978.693348059"
Sep 18 18:55:45.815409 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:45.815314    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:55:47.667107 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:47.667077    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:55:56.773724 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.773689    2558 generic.go:356] "Generic (PLEG): container finished" podID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerID="69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4" exitCode=0
Sep 18 18:55:56.774163 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.773756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerDied","Data":"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"}
Sep 18 18:55:56.774222 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.774187    2558 scope.go:117] "RemoveContainer" containerID="69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"
Sep 18 18:55:56.775395 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.775363    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:56.775477 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.775430    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:56.775477 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:56.775461    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:55:57.780003 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:57.779968    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerStarted","Data":"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"}
Sep 18 18:55:58.240642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:58.240542    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:55:58.241787 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:58.241760    2558 patch_prober.go:28] interesting pod/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 18 18:55:58.241849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:58.241821    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 18 18:55:59.240577 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:59.240543    2558 patch_prober.go:28] interesting pod/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 18 18:55:59.240996 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:55:59.240603    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 18 18:56:00.240077 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:00.240039    2558 patch_prober.go:28] interesting pod/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 18 18:56:00.240244 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:00.240110    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 18 18:56:01.240774 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:01.240737    2558 patch_prober.go:28] interesting pod/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 18 18:56:01.241166 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:01.240788    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 18 18:56:01.634518 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:01.634484    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:56:02.240547 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:02.240507    2558 patch_prober.go:28] interesting pod/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 18 18:56:02.240749 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:02.240572    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 18 18:56:03.244373 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:03.244344    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:56:03.245322 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:03.245302    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:56:04.374673 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.374641    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:56:04.375147 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.374939    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" podUID="3a266793-80d3-4cc3-b38d-6811090f8fa8" containerName="payload-processing" containerID="cri-o://5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12" gracePeriod=30
Sep 18 18:56:04.626706 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.626615    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:56:04.716957 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.716923    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:56:04.731457 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.731419    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume\") pod \"3a266793-80d3-4cc3-b38d-6811090f8fa8\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") "
Sep 18 18:56:04.731654 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.731554    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w9mb\" (UniqueName: \"kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb\") pod \"3a266793-80d3-4cc3-b38d-6811090f8fa8\" (UID: \"3a266793-80d3-4cc3-b38d-6811090f8fa8\") "
Sep 18 18:56:04.732506 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.732482    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3a266793-80d3-4cc3-b38d-6811090f8fa8" (UID: "3a266793-80d3-4cc3-b38d-6811090f8fa8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:56:04.734994 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.734968    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb" (OuterVolumeSpecName: "kube-api-access-7w9mb") pod "3a266793-80d3-4cc3-b38d-6811090f8fa8" (UID: "3a266793-80d3-4cc3-b38d-6811090f8fa8"). InnerVolumeSpecName "kube-api-access-7w9mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:56:04.828916 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.827754    2558 generic.go:356] "Generic (PLEG): container finished" podID="3a266793-80d3-4cc3-b38d-6811090f8fa8" containerID="5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12" exitCode=0
Sep 18 18:56:04.828916 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.828089    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy" containerID="cri-o://9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572" gracePeriod=30
Sep 18 18:56:04.828916 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.828249    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"
Sep 18 18:56:04.832633 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.832191    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" event={"ID":"3a266793-80d3-4cc3-b38d-6811090f8fa8","Type":"ContainerDied","Data":"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"}
Sep 18 18:56:04.832633 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.832257    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l" event={"ID":"3a266793-80d3-4cc3-b38d-6811090f8fa8","Type":"ContainerDied","Data":"444751ff7a411c8840e9843acb3df2cb107690dd870bc9ea9cd05c35fe3c8451"}
Sep 18 18:56:04.832633 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.832293    2558 scope.go:117] "RemoveContainer" containerID="5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"
Sep 18 18:56:04.833359 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.833336    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w9mb\" (UniqueName: \"kubernetes.io/projected/3a266793-80d3-4cc3-b38d-6811090f8fa8-kube-api-access-7w9mb\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.833512 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.833495    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a266793-80d3-4cc3-b38d-6811090f8fa8-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.847676 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.847098    2558 scope.go:117] "RemoveContainer" containerID="5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"
Sep 18 18:56:04.847758 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:56:04.847708    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12\": container with ID starting with 5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12 not found: ID does not exist" containerID="5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"
Sep 18 18:56:04.847861 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.847759    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12"} err="failed to get container status \"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12\": rpc error: code = NotFound desc = could not find container \"5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12\": container with ID starting with 5be140a0fe2df093e05f3b6e62ba605533f179daa248b92ca8df12ba08162e12 not found: ID does not exist"
Sep 18 18:56:04.885743 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.885654    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:56:04.892739 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:04.892708    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d2dcf91d-7c4b4469f7-pmm9l"]
Sep 18 18:56:05.436828 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:05.436794    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a266793-80d3-4cc3-b38d-6811090f8fa8" path="/var/lib/kubelet/pods/3a266793-80d3-4cc3-b38d-6811090f8fa8/volumes"
Sep 18 18:56:10.100276 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.100249    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:56:10.180985 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.180876    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.180985 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.180948    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.180985 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.180978    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.180998    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181020    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbn9c\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181151    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181211    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181283    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181305    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs\") pod \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\" (UID: \"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912\") "
Sep 18 18:56:10.181452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181324    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:56:10.181452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181341    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data" (OuterVolumeSpecName: "istio-data") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:56:10.181452 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181433    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:56:10.181694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181523    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:56:10.181694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181666    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:56:10.181784 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181752    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istiod-ca-cert\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.181784 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181769    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-credential-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.181857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181784    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-data\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.181857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181797    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.181857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.181810    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-workload-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.183430 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.183409    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:56:10.183512 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.183458    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:56:10.183922 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.183884    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c" (OuterVolumeSpecName: "kube-api-access-wbn9c") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "kube-api-access-wbn9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:56:10.184022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.183932    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token" (OuterVolumeSpecName: "istio-token") pod "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" (UID: "d643b5e2-d0ba-4b67-ab88-fa97e2ccc912"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:56:10.282597 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.282562    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbn9c\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-kube-api-access-wbn9c\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.282597 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.282593    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-envoy\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.282597 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.282606    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.282832 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.282615    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912-istio-podinfo\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:56:10.856618 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.856589    2558 generic.go:356] "Generic (PLEG): container finished" podID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerID="9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572" exitCode=0
Sep 18 18:56:10.856791 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.856638    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"
Sep 18 18:56:10.856791 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.856678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerDied","Data":"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"}
Sep 18 18:56:10.856791 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.856741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2" event={"ID":"d643b5e2-d0ba-4b67-ab88-fa97e2ccc912","Type":"ContainerDied","Data":"107ffa65636475cd26f035e2dc59f9ae86ccede795df2aecb6acc2a51ed2074d"}
Sep 18 18:56:10.856791 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.856773    2558 scope.go:117] "RemoveContainer" containerID="9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"
Sep 18 18:56:10.865794 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.865779    2558 scope.go:117] "RemoveContainer" containerID="69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"
Sep 18 18:56:10.873766 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.873751    2558 scope.go:117] "RemoveContainer" containerID="9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"
Sep 18 18:56:10.874086 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:56:10.874067    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572\": container with ID starting with 9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572 not found: ID does not exist" containerID="9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"
Sep 18 18:56:10.874152 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.874095    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572"} err="failed to get container status \"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572\": rpc error: code = NotFound desc = could not find container \"9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572\": container with ID starting with 9bab37172c92fc52282028fd60bd4f8fed2ad3feb34758848935479f800d0572 not found: ID does not exist"
Sep 18 18:56:10.874152 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.874113    2558 scope.go:117] "RemoveContainer" containerID="69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"
Sep 18 18:56:10.874368 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:56:10.874351    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4\": container with ID starting with 69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4 not found: ID does not exist" containerID="69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"
Sep 18 18:56:10.874408 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.874374    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4"} err="failed to get container status \"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4\": rpc error: code = NotFound desc = could not find container \"69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4\": container with ID starting with 69e88cd12408fade709837d4d91e07d3aa2d06f0cbb700d1f83ca92da596d5d4 not found: ID does not exist"
Sep 18 18:56:10.877786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.877763    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:56:10.883695 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:10.883675    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d2dcf91d-openshift-default-5bf588b496-h78c2"]
Sep 18 18:56:11.434021 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:11.433987    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" path="/var/lib/kubelet/pods/d643b5e2-d0ba-4b67-ab88-fa97e2ccc912/volumes"
Sep 18 18:56:12.206379 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:12.206345    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:56:32.718378 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:56:32.718340    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:57:01.404869 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:01.404832    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:57:13.950191 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.949938    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950465    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950481    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950495    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a266793-80d3-4cc3-b38d-6811090f8fa8" containerName="payload-processing"
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950504    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a266793-80d3-4cc3-b38d-6811090f8fa8" containerName="payload-processing"
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950605    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:13.950670 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.950621    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a266793-80d3-4cc3-b38d-6811090f8fa8" containerName="payload-processing"
Sep 18 18:57:13.953683 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.953662    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:13.957021 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.956985    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"default-dockercfg-2khll\""
Sep 18 18:57:13.957021 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.957021    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"distinct-2-w3-9d9f36-kserve-self-signed-certs\""
Sep 18 18:57:13.957270 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.957069    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"openshift-service-ca.crt\""
Sep 18 18:57:13.957270 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.957182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"kube-root-ca.crt\""
Sep 18 18:57:13.962242 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:13.962209    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 18:57:14.054478 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.054478 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.054694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054539    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.054694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054556    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7qss\" (UniqueName: \"kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.054694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.054694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.054643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.155800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.155800 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155801    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7qss\" (UniqueName: \"kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.155998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156282 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.156236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156335 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.156287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.156381 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.156369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.158397 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.158367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.158609 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.158588    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.164356 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.164333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7qss\" (UniqueName: \"kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss\") pod \"distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.265064 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.264976    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:14.534758 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:14.534732    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 18:57:14.537126 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:57:14.537098    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cd64820_7840_47a7_9a63_3d817e153754.slice/crio-24093912389bebe8037ef4fede86ca0894f24124b90455d5a23600ce06607e46 WatchSource:0}: Error finding container 24093912389bebe8037ef4fede86ca0894f24124b90455d5a23600ce06607e46: Status 404 returned error can't find the container with id 24093912389bebe8037ef4fede86ca0894f24124b90455d5a23600ce06607e46
Sep 18 18:57:15.148110 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:15.148067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" event={"ID":"9cd64820-7840-47a7-9a63-3d817e153754","Type":"ContainerStarted","Data":"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"}
Sep 18 18:57:15.148606 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:15.148120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" event={"ID":"9cd64820-7840-47a7-9a63-3d817e153754","Type":"ContainerStarted","Data":"24093912389bebe8037ef4fede86ca0894f24124b90455d5a23600ce06607e46"}
Sep 18 18:57:15.166430 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:15.166380    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" podStartSLOduration=2.166362442 podStartE2EDuration="2.166362442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:57:15.164673263 +0000 UTC m=+1078.184469902" watchObservedRunningTime="2026-09-18 18:57:15.166362442 +0000 UTC m=+1078.186159133"
Sep 18 18:57:20.729882 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.729842    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 18:57:20.730445 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.730430    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:20.730517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.730448    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:20.730729 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.730704    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d643b5e2-d0ba-4b67-ab88-fa97e2ccc912" containerName="istio-proxy"
Sep 18 18:57:20.734115 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.734090    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.736644 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.736624    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc289c\"/\"openshift-service-ca.crt\""
Sep 18 18:57:20.737259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.737242    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc289c\"/\"kube-root-ca.crt\""
Sep 18 18:57:20.737259 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.737253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc289c\"/\"default-dockercfg-t552s\""
Sep 18 18:57:20.737405 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.737253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc289c\"/\"distinct-w0-cc289c-kserve-self-signed-certs\""
Sep 18 18:57:20.743393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.743368    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 18:57:20.816188 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.816372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.816372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.816372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.816372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.816372 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.816347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb5vf\" (UniqueName: \"kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917300 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917300 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917550 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917550 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917550 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917550 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb5vf\" (UniqueName: \"kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917755 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.917849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.917846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.919827 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.919808    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.920107 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.920091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:20.925975 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:20.925950    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb5vf\" (UniqueName: \"kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf\") pod \"distinct-w0-cc289c-kserve-58c8bdd78c-lbndb\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") " pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:21.045089 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:21.045052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:21.318967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:21.318936    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 18:57:21.321554 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:57:21.321525    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddddbc95a_b594_4ecf_a974_19b7e0afabf4.slice/crio-2f11d9e48f6917e1cf816f8c261fb2f65219b7e041b53a479bed17e661df6445 WatchSource:0}: Error finding container 2f11d9e48f6917e1cf816f8c261fb2f65219b7e041b53a479bed17e661df6445: Status 404 returned error can't find the container with id 2f11d9e48f6917e1cf816f8c261fb2f65219b7e041b53a479bed17e661df6445
Sep 18 18:57:22.182000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:22.181956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" event={"ID":"dddbc95a-b594-4ecf-a974-19b7e0afabf4","Type":"ContainerStarted","Data":"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"}
Sep 18 18:57:22.182000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:22.182007    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" event={"ID":"dddbc95a-b594-4ecf-a974-19b7e0afabf4","Type":"ContainerStarted","Data":"2f11d9e48f6917e1cf816f8c261fb2f65219b7e041b53a479bed17e661df6445"}
Sep 18 18:57:22.198117 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:22.198059    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" podStartSLOduration=2.198042179 podStartE2EDuration="2.198042179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:57:22.197399516 +0000 UTC m=+1085.217196146" watchObservedRunningTime="2026-09-18 18:57:22.198042179 +0000 UTC m=+1085.217838804"
Sep 18 18:57:24.266107 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:24.266072    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:24.266489 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:24.266123    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:24.278824 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:24.278784    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:25.208581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:25.208553    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 18:57:31.045676 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:31.045625    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:31.046121 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:31.045701    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:31.058822 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:31.058795    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:31.239313 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:31.239281    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 18:57:31.823299 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:31.823265    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:57:47.251603 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.251563    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:57:47.255210 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.255186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.258913 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.258872    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-62de055a-dockercfg-8bzc7\""
Sep 18 18:57:47.259042 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.258986    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-62de055a\""
Sep 18 18:57:47.267396 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.267355    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:57:47.291799 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.291769    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:57:47.295178 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.295162    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.305156 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.305130    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:57:47.367277 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.367236    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4h2f\" (UniqueName: \"kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.367642 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.367620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.468592 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.468549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.468765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.468638    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.468765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.468684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4h2f\" (UniqueName: \"kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.468765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.468706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9z47\" (UniqueName: \"kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.469451 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.469431    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.477057 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.477023    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4h2f\" (UniqueName: \"kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f\") pod \"payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.567076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.567040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:47.569413 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.569385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9z47\" (UniqueName: \"kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.569466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.569431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.570382 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.570293    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.578334 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.578312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9z47\" (UniqueName: \"kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47\") pod \"payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") " pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.606704 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.606672    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:47.849256 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.849232    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:57:47.851383 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:57:47.851358    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591984d8_e69a_4fdd_9d43_3a053d8b27db.slice/crio-6416a1a1614dbe8cb8e423b3e08ea8b573391df95ac89f923039ff075d71067e WatchSource:0}: Error finding container 6416a1a1614dbe8cb8e423b3e08ea8b573391df95ac89f923039ff075d71067e: Status 404 returned error can't find the container with id 6416a1a1614dbe8cb8e423b3e08ea8b573391df95ac89f923039ff075d71067e
Sep 18 18:57:47.894547 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:47.894523    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:57:47.904002 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:57:47.903976    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18757dd6_3723_4a71_af2b_2bca2f705d16.slice/crio-320ac4aaebe03557eda89dc4e13b3fca702424b149e282fdcf9cc4d790b63edc WatchSource:0}: Error finding container 320ac4aaebe03557eda89dc4e13b3fca702424b149e282fdcf9cc4d790b63edc: Status 404 returned error can't find the container with id 320ac4aaebe03557eda89dc4e13b3fca702424b149e282fdcf9cc4d790b63edc
Sep 18 18:57:48.311010 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.310969    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" event={"ID":"591984d8-e69a-4fdd-9d43-3a053d8b27db","Type":"ContainerStarted","Data":"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"}
Sep 18 18:57:48.311511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.311038    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" event={"ID":"591984d8-e69a-4fdd-9d43-3a053d8b27db","Type":"ContainerStarted","Data":"6416a1a1614dbe8cb8e423b3e08ea8b573391df95ac89f923039ff075d71067e"}
Sep 18 18:57:48.311662 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.311638    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:57:48.313573 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.313540    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" event={"ID":"18757dd6-3723-4a71-af2b-2bca2f705d16","Type":"ContainerStarted","Data":"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"}
Sep 18 18:57:48.313702 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.313592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" event={"ID":"18757dd6-3723-4a71-af2b-2bca2f705d16","Type":"ContainerStarted","Data":"320ac4aaebe03557eda89dc4e13b3fca702424b149e282fdcf9cc4d790b63edc"}
Sep 18 18:57:48.313702 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.313657    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:57:48.334956 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.334874    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" podStartSLOduration=1.3348569860000001 podStartE2EDuration="1.334856986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:57:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:57:48.332578516 +0000 UTC m=+1111.352375168" watchObservedRunningTime="2026-09-18 18:57:48.334856986 +0000 UTC m=+1111.354653611"
Sep 18 18:57:48.349119 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:48.349063    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" podStartSLOduration=1.349045942 podStartE2EDuration="1.349045942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:57:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:57:48.348076088 +0000 UTC m=+1111.367872717" watchObservedRunningTime="2026-09-18 18:57:48.349045942 +0000 UTC m=+1111.368842568"
Sep 18 18:57:59.321211 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:57:59.321180    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:58:00.326466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:00.326439    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:58:07.904143 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:07.904096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:58:16.030599 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.030561    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:58:16.030993 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.030874    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" podUID="44b40a66-ff53-4611-8310-145e6798e36a" containerName="payload-processing" containerID="cri-o://e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e" gracePeriod=30
Sep 18 18:58:16.035281 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.035251    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:58:16.035544 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.035492    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" podUID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" containerName="payload-pre-processing" containerID="cri-o://06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4" gracePeriod=30
Sep 18 18:58:16.290656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.290588    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:58:16.323208 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.323186    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:58:16.450770 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.450734    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4ksb\" (UniqueName: \"kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb\") pod \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") "
Sep 18 18:58:16.450967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.450794    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt4xd\" (UniqueName: \"kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd\") pod \"44b40a66-ff53-4611-8310-145e6798e36a\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") "
Sep 18 18:58:16.450967 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.450831    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume\") pod \"44b40a66-ff53-4611-8310-145e6798e36a\" (UID: \"44b40a66-ff53-4611-8310-145e6798e36a\") "
Sep 18 18:58:16.451184 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.451160    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume\") pod \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\" (UID: \"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4\") "
Sep 18 18:58:16.451410 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.451384    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44b40a66-ff53-4611-8310-145e6798e36a" (UID: "44b40a66-ff53-4611-8310-145e6798e36a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:16.451526 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.451478    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44b40a66-ff53-4611-8310-145e6798e36a-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.451592 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.451520    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" (UID: "d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:16.452086 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.452061    2558 generic.go:356] "Generic (PLEG): container finished" podID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" containerID="06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4" exitCode=0
Sep 18 18:58:16.452162 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.452146    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"
Sep 18 18:58:16.452207 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.452149    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" event={"ID":"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4","Type":"ContainerDied","Data":"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"}
Sep 18 18:58:16.452267 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.452248    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv" event={"ID":"d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4","Type":"ContainerDied","Data":"2d404c787fa23a226f360cb0dee252467e58610126b5f6391efc68bcae54da49"}
Sep 18 18:58:16.452326 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.452285    2558 scope.go:117] "RemoveContainer" containerID="06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"
Sep 18 18:58:16.453589 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453568    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd" (OuterVolumeSpecName: "kube-api-access-rt4xd") pod "44b40a66-ff53-4611-8310-145e6798e36a" (UID: "44b40a66-ff53-4611-8310-145e6798e36a"). InnerVolumeSpecName "kube-api-access-rt4xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:16.453668 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453609    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb" (OuterVolumeSpecName: "kube-api-access-g4ksb") pod "d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" (UID: "d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4"). InnerVolumeSpecName "kube-api-access-g4ksb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:16.453763 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453741    2558 generic.go:356] "Generic (PLEG): container finished" podID="44b40a66-ff53-4611-8310-145e6798e36a" containerID="e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e" exitCode=0
Sep 18 18:58:16.453816 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" event={"ID":"44b40a66-ff53-4611-8310-145e6798e36a","Type":"ContainerDied","Data":"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"}
Sep 18 18:58:16.453816 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453796    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"
Sep 18 18:58:16.453816 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.453803    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg" event={"ID":"44b40a66-ff53-4611-8310-145e6798e36a","Type":"ContainerDied","Data":"b974102819a14b1b2ec1a2e834a10ef63e43c837e277dd5173204dec17e9d30e"}
Sep 18 18:58:16.461694 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.461675    2558 scope.go:117] "RemoveContainer" containerID="06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"
Sep 18 18:58:16.462125 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:16.462099    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4\": container with ID starting with 06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4 not found: ID does not exist" containerID="06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"
Sep 18 18:58:16.462207 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.462135    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4"} err="failed to get container status \"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4\": rpc error: code = NotFound desc = could not find container \"06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4\": container with ID starting with 06cc143b03d793380d179e531aa92a6c6a53b8de3185972aed2f81e9a9f890b4 not found: ID does not exist"
Sep 18 18:58:16.462207 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.462172    2558 scope.go:117] "RemoveContainer" containerID="e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"
Sep 18 18:58:16.469946 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.469927    2558 scope.go:117] "RemoveContainer" containerID="e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"
Sep 18 18:58:16.470236 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:16.470216    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e\": container with ID starting with e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e not found: ID does not exist" containerID="e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"
Sep 18 18:58:16.470299 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.470242    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e"} err="failed to get container status \"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e\": rpc error: code = NotFound desc = could not find container \"e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e\": container with ID starting with e592bb822cc338347ca738c191a64c50837587469ad763a2c942659b5f01471e not found: ID does not exist"
Sep 18 18:58:16.478517 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.478492    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:58:16.482121 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.482100    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bcb770-7fc555fbdc-6jxfg"]
Sep 18 18:58:16.552313 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.552226    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.552313 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.552252    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4ksb\" (UniqueName: \"kubernetes.io/projected/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4-kube-api-access-g4ksb\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.552313 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.552265    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt4xd\" (UniqueName: \"kubernetes.io/projected/44b40a66-ff53-4611-8310-145e6798e36a-kube-api-access-rt4xd\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.775335 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.775305    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:58:16.777130 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:16.777106    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bcb770-7c7bf5d4db-892gv"]
Sep 18 18:58:17.435118 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:17.435041    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44b40a66-ff53-4611-8310-145e6798e36a" path="/var/lib/kubelet/pods/44b40a66-ff53-4611-8310-145e6798e36a/volumes"
Sep 18 18:58:17.435447 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:17.435411    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" path="/var/lib/kubelet/pods/d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4/volumes"
Sep 18 18:58:22.996646 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:22.996610    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:58:22.997172 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:22.996877    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy" containerID="cri-o://e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771" gracePeriod=30
Sep 18 18:58:28.241198 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.241174    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:58:28.365422 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365386    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365422 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365421    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365437    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365470    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czzt5\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365565    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365653 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365674    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365705    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo\") pod \"03abdc1a-6279-44dd-b273-02bb393e8f25\" (UID: \"03abdc1a-6279-44dd-b273-02bb393e8f25\") "
Sep 18 18:58:28.365926 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365736    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data" (OuterVolumeSpecName: "istio-data") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:28.366080 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365869    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:28.366080 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365945    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:28.366080 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.365981    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:28.366197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.366111    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-data\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.366197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.366128    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03abdc1a-6279-44dd-b273-02bb393e8f25-istiod-ca-cert\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.366197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.366140    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.366197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.366153    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-credential-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.366197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.366136    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:28.367970 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.367938    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token" (OuterVolumeSpecName: "istio-token") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:28.368077 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.368017    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:58:28.368254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.368230    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5" (OuterVolumeSpecName: "kube-api-access-czzt5") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "kube-api-access-czzt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:28.368254 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.368238    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03abdc1a-6279-44dd-b273-02bb393e8f25" (UID: "03abdc1a-6279-44dd-b273-02bb393e8f25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:28.467022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.466991    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-podinfo\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.467022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.467019    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-envoy\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.467022 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.467028    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czzt5\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-kube-api-access-czzt5\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.467251 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.467037    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03abdc1a-6279-44dd-b273-02bb393e8f25-istio-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.467251 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.467046    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03abdc1a-6279-44dd-b273-02bb393e8f25-workload-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:28.512584 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.512546    2558 generic.go:356] "Generic (PLEG): container finished" podID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerID="e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771" exitCode=0
Sep 18 18:58:28.512767 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.512608    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"
Sep 18 18:58:28.512767 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.512635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerDied","Data":"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"}
Sep 18 18:58:28.512767 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.512691    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k" event={"ID":"03abdc1a-6279-44dd-b273-02bb393e8f25","Type":"ContainerDied","Data":"ddba79f29c41f5cf83590f97c086bb261c93dc36a355d1e141ebf9b1aa50e134"}
Sep 18 18:58:28.512767 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.512714    2558 scope.go:117] "RemoveContainer" containerID="e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"
Sep 18 18:58:28.521803 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.521785    2558 scope.go:117] "RemoveContainer" containerID="36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"
Sep 18 18:58:28.530146 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.530113    2558 scope.go:117] "RemoveContainer" containerID="e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"
Sep 18 18:58:28.530431 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:28.530411    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771\": container with ID starting with e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771 not found: ID does not exist" containerID="e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"
Sep 18 18:58:28.530481 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.530443    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771"} err="failed to get container status \"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771\": rpc error: code = NotFound desc = could not find container \"e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771\": container with ID starting with e27ea1ffe4f74bcae17623de49c055da53a3151f147a71cf1d5f2b9debd42771 not found: ID does not exist"
Sep 18 18:58:28.530481 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.530462    2558 scope.go:117] "RemoveContainer" containerID="36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"
Sep 18 18:58:28.530707 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:28.530690    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec\": container with ID starting with 36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec not found: ID does not exist" containerID="36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"
Sep 18 18:58:28.530755 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.530713    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec"} err="failed to get container status \"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec\": rpc error: code = NotFound desc = could not find container \"36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec\": container with ID starting with 36c39e0a66d8a6584f9144f335357d9e1a62a765e45ca3bc6d9b56d2a82f86ec not found: ID does not exist"
Sep 18 18:58:28.533689 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.533663    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:58:28.538229 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:28.538206    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bcb770-openshift-default-64fb9d4f9b-zrg8k"]
Sep 18 18:58:29.433546 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:29.433513    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" path="/var/lib/kubelet/pods/03abdc1a-6279-44dd-b273-02bb393e8f25/volumes"
Sep 18 18:58:31.882693 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:31.877357    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:58:31.882693 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:31.877666    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" podUID="18757dd6-3723-4a71-af2b-2bca2f705d16" containerName="payload-processing" containerID="cri-o://9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a" gracePeriod=30
Sep 18 18:58:31.895236 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:31.893497    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:58:31.895236 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:31.893814    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" podUID="591984d8-e69a-4fdd-9d43-3a053d8b27db" containerName="payload-pre-processing" containerID="cri-o://256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0" gracePeriod=30
Sep 18 18:58:32.152855 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.152831    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:58:32.183738 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.183710    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:58:32.304336 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304302    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume\") pod \"18757dd6-3723-4a71-af2b-2bca2f705d16\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") "
Sep 18 18:58:32.304336 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304338    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9z47\" (UniqueName: \"kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47\") pod \"18757dd6-3723-4a71-af2b-2bca2f705d16\" (UID: \"18757dd6-3723-4a71-af2b-2bca2f705d16\") "
Sep 18 18:58:32.304533 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304382    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume\") pod \"591984d8-e69a-4fdd-9d43-3a053d8b27db\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") "
Sep 18 18:58:32.304533 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304403    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4h2f\" (UniqueName: \"kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f\") pod \"591984d8-e69a-4fdd-9d43-3a053d8b27db\" (UID: \"591984d8-e69a-4fdd-9d43-3a053d8b27db\") "
Sep 18 18:58:32.304784 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18757dd6-3723-4a71-af2b-2bca2f705d16" (UID: "18757dd6-3723-4a71-af2b-2bca2f705d16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:32.304849 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.304757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "591984d8-e69a-4fdd-9d43-3a053d8b27db" (UID: "591984d8-e69a-4fdd-9d43-3a053d8b27db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:32.306648 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.306627    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47" (OuterVolumeSpecName: "kube-api-access-t9z47") pod "18757dd6-3723-4a71-af2b-2bca2f705d16" (UID: "18757dd6-3723-4a71-af2b-2bca2f705d16"). InnerVolumeSpecName "kube-api-access-t9z47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:32.306810 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.306794    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f" (OuterVolumeSpecName: "kube-api-access-g4h2f") pod "591984d8-e69a-4fdd-9d43-3a053d8b27db" (UID: "591984d8-e69a-4fdd-9d43-3a053d8b27db"). InnerVolumeSpecName "kube-api-access-g4h2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:32.405829 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.405750    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4h2f\" (UniqueName: \"kubernetes.io/projected/591984d8-e69a-4fdd-9d43-3a053d8b27db-kube-api-access-g4h2f\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.405829 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.405783    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18757dd6-3723-4a71-af2b-2bca2f705d16-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.405829 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.405794    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9z47\" (UniqueName: \"kubernetes.io/projected/18757dd6-3723-4a71-af2b-2bca2f705d16-kube-api-access-t9z47\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.405829 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.405803    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591984d8-e69a-4fdd-9d43-3a053d8b27db-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.533279 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.533241    2558 generic.go:356] "Generic (PLEG): container finished" podID="18757dd6-3723-4a71-af2b-2bca2f705d16" containerID="9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a" exitCode=0
Sep 18 18:58:32.533466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.533287    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"
Sep 18 18:58:32.533466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.533325    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" event={"ID":"18757dd6-3723-4a71-af2b-2bca2f705d16","Type":"ContainerDied","Data":"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"}
Sep 18 18:58:32.533466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.533386    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb" event={"ID":"18757dd6-3723-4a71-af2b-2bca2f705d16","Type":"ContainerDied","Data":"320ac4aaebe03557eda89dc4e13b3fca702424b149e282fdcf9cc4d790b63edc"}
Sep 18 18:58:32.533466 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.533409    2558 scope.go:117] "RemoveContainer" containerID="9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"
Sep 18 18:58:32.534688 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.534665    2558 generic.go:356] "Generic (PLEG): container finished" podID="591984d8-e69a-4fdd-9d43-3a053d8b27db" containerID="256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0" exitCode=0
Sep 18 18:58:32.534797 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.534708    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"
Sep 18 18:58:32.534797 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.534733    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" event={"ID":"591984d8-e69a-4fdd-9d43-3a053d8b27db","Type":"ContainerDied","Data":"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"}
Sep 18 18:58:32.534797 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.534775    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw" event={"ID":"591984d8-e69a-4fdd-9d43-3a053d8b27db","Type":"ContainerDied","Data":"6416a1a1614dbe8cb8e423b3e08ea8b573391df95ac89f923039ff075d71067e"}
Sep 18 18:58:32.543336 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.543231    2558 scope.go:117] "RemoveContainer" containerID="9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"
Sep 18 18:58:32.543626 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:32.543606    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a\": container with ID starting with 9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a not found: ID does not exist" containerID="9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"
Sep 18 18:58:32.543933 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.543634    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a"} err="failed to get container status \"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a\": rpc error: code = NotFound desc = could not find container \"9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a\": container with ID starting with 9a261343a013bce0fb57d8f2af09f7bd8f738e4e22074808a77f39de90b50c4a not found: ID does not exist"
Sep 18 18:58:32.543933 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.543654    2558 scope.go:117] "RemoveContainer" containerID="256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"
Sep 18 18:58:32.552518 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.552502    2558 scope.go:117] "RemoveContainer" containerID="256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"
Sep 18 18:58:32.552790 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:32.552774    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0\": container with ID starting with 256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0 not found: ID does not exist" containerID="256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"
Sep 18 18:58:32.552853 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.552796    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0"} err="failed to get container status \"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0\": rpc error: code = NotFound desc = could not find container \"256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0\": container with ID starting with 256a490d07c4767f86f70feadf0ce5a26c5a119fc7ed4bb01284aaabbbc28db0 not found: ID does not exist"
Sep 18 18:58:32.557913 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.557866    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:58:32.567161 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.567138    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-62de055a-54f8d487d6-2n5hw"]
Sep 18 18:58:32.578390 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.578365    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:58:32.583975 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:32.583951    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-62de055a-648bd87c8f-xk9nb"]
Sep 18 18:58:33.433757 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:33.433725    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18757dd6-3723-4a71-af2b-2bca2f705d16" path="/var/lib/kubelet/pods/18757dd6-3723-4a71-af2b-2bca2f705d16/volumes"
Sep 18 18:58:33.434166 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:33.434152    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="591984d8-e69a-4fdd-9d43-3a053d8b27db" path="/var/lib/kubelet/pods/591984d8-e69a-4fdd-9d43-3a053d8b27db/volumes"
Sep 18 18:58:34.447629 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.447598    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:58:34.448115 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.447801    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" podUID="b7c54852-7b7c-430f-994b-bd1490d243db" containerName="maas-api" containerID="cri-o://8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8" gracePeriod=30
Sep 18 18:58:34.506177 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.506128    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:58:34.506407 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.506380    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" podUID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" containerName="payload-processing" containerID="cri-o://89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d" gracePeriod=30
Sep 18 18:58:34.780458 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.780432    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:58:34.907250 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.907227    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:58:34.929109 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929083    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7b5t\" (UniqueName: \"kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t\") pod \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") "
Sep 18 18:58:34.929280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929127    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs\") pod \"b7c54852-7b7c-430f-994b-bd1490d243db\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") "
Sep 18 18:58:34.929280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929176    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7jgx\" (UniqueName: \"kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx\") pod \"b7c54852-7b7c-430f-994b-bd1490d243db\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") "
Sep 18 18:58:34.929280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929193    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca\") pod \"b7c54852-7b7c-430f-994b-bd1490d243db\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") "
Sep 18 18:58:34.929280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929215    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls\") pod \"b7c54852-7b7c-430f-994b-bd1490d243db\" (UID: \"b7c54852-7b7c-430f-994b-bd1490d243db\") "
Sep 18 18:58:34.929280 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929252    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume\") pod \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\" (UID: \"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0\") "
Sep 18 18:58:34.929626 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929582    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7c54852-7b7c-430f-994b-bd1490d243db" (UID: "b7c54852-7b7c-430f-994b-bd1490d243db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:34.930011 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.929812    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" (UID: "830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:34.931761 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.931728    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t" (OuterVolumeSpecName: "kube-api-access-q7b5t") pod "830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" (UID: "830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0"). InnerVolumeSpecName "kube-api-access-q7b5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:34.931877 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.931826    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx" (OuterVolumeSpecName: "kube-api-access-x7jgx") pod "b7c54852-7b7c-430f-994b-bd1490d243db" (UID: "b7c54852-7b7c-430f-994b-bd1490d243db"). InnerVolumeSpecName "kube-api-access-x7jgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:34.932133 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.932112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7c54852-7b7c-430f-994b-bd1490d243db" (UID: "b7c54852-7b7c-430f-994b-bd1490d243db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:34.932265 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:34.932245    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b7c54852-7b7c-430f-994b-bd1490d243db" (UID: "b7c54852-7b7c-430f-994b-bd1490d243db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:35.030625 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030591    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7b5t\" (UniqueName: \"kubernetes.io/projected/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-kube-api-access-q7b5t\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.030625 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030623    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.030625 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030632    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7jgx\" (UniqueName: \"kubernetes.io/projected/b7c54852-7b7c-430f-994b-bd1490d243db-kube-api-access-x7jgx\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.030857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030641    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c54852-7b7c-430f-994b-bd1490d243db-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.030857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030650    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c54852-7b7c-430f-994b-bd1490d243db-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.030857 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.030657    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.552018 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.551985    2558 generic.go:356] "Generic (PLEG): container finished" podID="b7c54852-7b7c-430f-994b-bd1490d243db" containerID="8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8" exitCode=0
Sep 18 18:58:35.552464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.552030    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"
Sep 18 18:58:35.552464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.552061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" event={"ID":"b7c54852-7b7c-430f-994b-bd1490d243db","Type":"ContainerDied","Data":"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"}
Sep 18 18:58:35.552464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.552121    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f" event={"ID":"b7c54852-7b7c-430f-994b-bd1490d243db","Type":"ContainerDied","Data":"177ae93c16f30d5c2d2b32b54208a306b42d7125bbe9ab65a9e195deffb9f937"}
Sep 18 18:58:35.552464 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.552156    2558 scope.go:117] "RemoveContainer" containerID="8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"
Sep 18 18:58:35.553467 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.553447    2558 generic.go:356] "Generic (PLEG): container finished" podID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" containerID="89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d" exitCode=0
Sep 18 18:58:35.553545 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.553484    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" event={"ID":"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0","Type":"ContainerDied","Data":"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"}
Sep 18 18:58:35.553545 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.553494    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"
Sep 18 18:58:35.553545 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.553522    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv" event={"ID":"830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0","Type":"ContainerDied","Data":"da09cd8da1e84d1cd294df2153d8eb65498000d6c0a8d8d88419d80b41b59155"}
Sep 18 18:58:35.561604 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.561583    2558 scope.go:117] "RemoveContainer" containerID="8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"
Sep 18 18:58:35.561879 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:35.561862    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8\": container with ID starting with 8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8 not found: ID does not exist" containerID="8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"
Sep 18 18:58:35.561970 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.561909    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8"} err="failed to get container status \"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8\": rpc error: code = NotFound desc = could not find container \"8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8\": container with ID starting with 8183814b06531ea77e10829f6fa0aa3aed25e25534e8649c4abb949a24ec45a8 not found: ID does not exist"
Sep 18 18:58:35.561970 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.561931    2558 scope.go:117] "RemoveContainer" containerID="89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"
Sep 18 18:58:35.570024 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.570003    2558 scope.go:117] "RemoveContainer" containerID="89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"
Sep 18 18:58:35.570324 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:35.570304    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d\": container with ID starting with 89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d not found: ID does not exist" containerID="89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"
Sep 18 18:58:35.570395 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.570335    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d"} err="failed to get container status \"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d\": rpc error: code = NotFound desc = could not find container \"89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d\": container with ID starting with 89bd5d9fde5939915652243497ab61ece6575484e68aadf39744303a3a71b59d not found: ID does not exist"
Sep 18 18:58:35.570520 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.570501    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:58:35.573985 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.573965    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-897f61-586f9f8557-j4r9f"]
Sep 18 18:58:35.582935 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.582911    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:58:35.587183 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:35.587163    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-897f61-7ccb4f565-qp6hv"]
Sep 18 18:58:37.434364 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:37.434329    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" path="/var/lib/kubelet/pods/830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0/volumes"
Sep 18 18:58:37.434725 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:37.434696    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c54852-7b7c-430f-994b-bd1490d243db" path="/var/lib/kubelet/pods/b7c54852-7b7c-430f-994b-bd1490d243db/volumes"
Sep 18 18:58:41.975340 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975253    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 18:58:41.975738 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975721    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" containerName="payload-pre-processing"
Sep 18 18:58:41.975782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975743    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" containerName="payload-pre-processing"
Sep 18 18:58:41.975782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975756    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c54852-7b7c-430f-994b-bd1490d243db" containerName="maas-api"
Sep 18 18:58:41.975782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975764    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c54852-7b7c-430f-994b-bd1490d243db" containerName="maas-api"
Sep 18 18:58:41.975782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975777    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18757dd6-3723-4a71-af2b-2bca2f705d16" containerName="payload-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975786    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="18757dd6-3723-4a71-af2b-2bca2f705d16" containerName="payload-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975798    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975806    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975829    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b40a66-ff53-4611-8310-145e6798e36a" containerName="payload-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975838    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b40a66-ff53-4611-8310-145e6798e36a" containerName="payload-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975848    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591984d8-e69a-4fdd-9d43-3a053d8b27db" containerName="payload-pre-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975856    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="591984d8-e69a-4fdd-9d43-3a053d8b27db" containerName="payload-pre-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975871    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" containerName="payload-processing"
Sep 18 18:58:41.975919 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.975880    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" containerName="payload-processing"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976000    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976012    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b40a66-ff53-4611-8310-145e6798e36a" containerName="payload-processing"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976022    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c54852-7b7c-430f-994b-bd1490d243db" containerName="maas-api"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976031    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="18757dd6-3723-4a71-af2b-2bca2f705d16" containerName="payload-processing"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976038    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4343e28-58c3-4c1a-ab67-02d9a4b7c4f4" containerName="payload-pre-processing"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976045    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="591984d8-e69a-4fdd-9d43-3a053d8b27db" containerName="payload-pre-processing"
Sep 18 18:58:41.976383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.976055    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="830a2656-fe5b-44f4-93ba-dd3d7c6eb5e0" containerName="payload-processing"
Sep 18 18:58:41.980905 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.980870    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:41.982972 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.982952    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-298125-openshift-default-dockercfg-kgmfp\""
Sep 18 18:58:41.989262 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:41.989236    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 18:58:42.088772 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.088772 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089015 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088797    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089015 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088881    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089015 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089015 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.088962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089180 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.089033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089180 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.089067    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rw6f\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.089180 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.089121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190000 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.189960    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190053    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190072    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190187 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190502 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190502 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rw6f\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190617 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.190990 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.190958    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.191095 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.191025    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.191170 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.191136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.191223 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.191183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.192966 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.192937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.193255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.193233    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.198711 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.198686    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.198812 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.198746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rw6f\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f\") pod \"e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") " pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.292423 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.292387    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:42.570828 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.570798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 18:58:42.573622 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:58:42.573591    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ee097d_9630_400b_a66f_0dde1e81333e.slice/crio-56601ef6220e306977af5280f8e438c16057986dff781743bf92012f66cb1bc1 WatchSource:0}: Error finding container 56601ef6220e306977af5280f8e438c16057986dff781743bf92012f66cb1bc1: Status 404 returned error can't find the container with id 56601ef6220e306977af5280f8e438c16057986dff781743bf92012f66cb1bc1
Sep 18 18:58:42.575875 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.575848    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:42.575965 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.575932    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:42.575999 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.575966    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:42.597801 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:42.597775    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerStarted","Data":"56601ef6220e306977af5280f8e438c16057986dff781743bf92012f66cb1bc1"}
Sep 18 18:58:43.604324 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:43.604285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerStarted","Data":"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"}
Sep 18 18:58:43.623333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:43.623286    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podStartSLOduration=2.623268949 podStartE2EDuration="2.623268949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:43.621837596 +0000 UTC m=+1166.641634232" watchObservedRunningTime="2026-09-18 18:58:43.623268949 +0000 UTC m=+1166.643065575"
Sep 18 18:58:44.292687 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:44.292650    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:44.298707 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:44.298679    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:44.609087 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:44.609006    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:44.609924 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:44.609875    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:58:47.308251 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.308211    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 18:58:47.308988 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.308964    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:47.308988 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.308991    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:47.309165 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.309152    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="03abdc1a-6279-44dd-b273-02bb393e8f25" containerName="istio-proxy"
Sep 18 18:58:47.312795 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.312771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.315232 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.315196    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-298125-serving-cert\""
Sep 18 18:58:47.324124 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.324094    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 18:58:47.339948 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.339923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.340095 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.339992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nbc2\" (UniqueName: \"kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.340095 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.340028    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.340095 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.340050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.441697 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.441650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.441920 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.441764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nbc2\" (UniqueName: \"kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.441920 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.441827    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.441920 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.441863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.442484 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:47.442293    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-serving-cert: secret "maas-api-e2e-shared-a-w2-298125-serving-cert" not found
Sep 18 18:58:47.442484 ip-10-0-132-229 kubenswrapper[2558]: E0918 18:58:47.442399    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls podName:44485db6-8f2a-4187-b46f-0540a303de0d nodeName:}" failed. No retries permitted until 2026-09-18 18:58:47.942376011 +0000 UTC m=+1170.962172616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls") pod "maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" (UID: "44485db6-8f2a-4187-b46f-0540a303de0d") : secret "maas-api-e2e-shared-a-w2-298125-serving-cert" not found
Sep 18 18:58:47.442982 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.442958    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.445370 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.445338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.453343 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.453320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nbc2\" (UniqueName: \"kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.946758 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.946719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:47.949424 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:47.949399    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:48.035498 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.035461    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 18:58:48.038825 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.038808    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.042274 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.042253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-298125-dockercfg-9xj86\""
Sep 18 18:58:48.043300 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.043283    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-298125\""
Sep 18 18:58:48.049206 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.049175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 18:58:48.147942 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.147883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.148089 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.147954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk7gb\" (UniqueName: \"kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.225337 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.225253    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:48.249506 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.249462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.249651 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.249516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk7gb\" (UniqueName: \"kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.250238 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.250217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.261928 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.261878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk7gb\" (UniqueName: \"kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb\") pod \"payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.351677 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.351636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:48.499098 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.499072    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 18:58:48.501687 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:58:48.501656    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44485db6_8f2a_4187_b46f_0540a303de0d.slice/crio-4e69ce5adbb4d63e0859ae61f009f25f2d2e7881688a67227733dac23cdc733d WatchSource:0}: Error finding container 4e69ce5adbb4d63e0859ae61f009f25f2d2e7881688a67227733dac23cdc733d: Status 404 returned error can't find the container with id 4e69ce5adbb4d63e0859ae61f009f25f2d2e7881688a67227733dac23cdc733d
Sep 18 18:58:48.503145 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.503129    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:58:48.627513 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.627474    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" event={"ID":"44485db6-8f2a-4187-b46f-0540a303de0d","Type":"ContainerStarted","Data":"4e69ce5adbb4d63e0859ae61f009f25f2d2e7881688a67227733dac23cdc733d"}
Sep 18 18:58:48.630320 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:48.630295    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 18:58:48.632535 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:58:48.632501    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3430d71e_bcf2_452c_b912_c5613fd0948a.slice/crio-0bf6bbc36a23c9b35031236cc2a4fb9d08ba8c60ace05147eafff5a17446917c WatchSource:0}: Error finding container 0bf6bbc36a23c9b35031236cc2a4fb9d08ba8c60ace05147eafff5a17446917c: Status 404 returned error can't find the container with id 0bf6bbc36a23c9b35031236cc2a4fb9d08ba8c60ace05147eafff5a17446917c
Sep 18 18:58:49.113535 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.113501    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:58:49.633838 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.633798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" event={"ID":"3430d71e-bcf2-452c-b912-c5613fd0948a","Type":"ContainerStarted","Data":"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"}
Sep 18 18:58:49.634333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.633856    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" event={"ID":"3430d71e-bcf2-452c-b912-c5613fd0948a","Type":"ContainerStarted","Data":"0bf6bbc36a23c9b35031236cc2a4fb9d08ba8c60ace05147eafff5a17446917c"}
Sep 18 18:58:49.634333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.633917    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:58:49.635442 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.635417    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" event={"ID":"44485db6-8f2a-4187-b46f-0540a303de0d","Type":"ContainerStarted","Data":"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"}
Sep 18 18:58:49.635566 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.635544    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:49.658333 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.658278    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" podStartSLOduration=1.658259887 podStartE2EDuration="1.658259887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:49.657556102 +0000 UTC m=+1172.677352728" watchObservedRunningTime="2026-09-18 18:58:49.658259887 +0000 UTC m=+1172.678056512"
Sep 18 18:58:49.683191 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:49.683139    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" podStartSLOduration=2.2709422200000002 podStartE2EDuration="2.683121575s" totalImagesPullingTime="412.179355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:49.682673957 +0000 UTC m=+1172.702470584" watchObservedRunningTime="2026-09-18 18:58:49.683121575 +0000 UTC m=+1172.702918199"
Sep 18 18:58:54.193872 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.193836    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 18:58:54.197538 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.197510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.203170 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.203142    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-880bee-openshift-default-dockercfg-sgwq5\""
Sep 18 18:58:54.213525 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.213495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 18:58:54.309153 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309238    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309362 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vcj7\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.309581 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.309462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410072 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410297 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410220    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vcj7\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410656 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410629    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410719 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410695    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410756 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.410993 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.411048 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.411003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.411048 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.410976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.411288 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.411268    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.412609 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.412587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.413326 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.413309    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.418847 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.418825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vcj7\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.419124 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.419104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token\") pod \"e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") " pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.509393 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.509303    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:54.794821 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.794794    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 18:58:54.796780 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:58:54.796740    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853c9fcd_c69a_40d4_aeab_2f7113077e37.slice/crio-5e616a1b2fbce994957a6c65a76bab4ef77e6507c82523b26915284166dfd557 WatchSource:0}: Error finding container 5e616a1b2fbce994957a6c65a76bab4ef77e6507c82523b26915284166dfd557: Status 404 returned error can't find the container with id 5e616a1b2fbce994957a6c65a76bab4ef77e6507c82523b26915284166dfd557
Sep 18 18:58:54.799075 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.799046    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:54.799146 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.799112    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:54.799146 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:54.799138    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:58:55.646450 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:55.646421    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 18:58:55.675244 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:55.675206    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerStarted","Data":"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"}
Sep 18 18:58:55.675445 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:55.675291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerStarted","Data":"5e616a1b2fbce994957a6c65a76bab4ef77e6507c82523b26915284166dfd557"}
Sep 18 18:58:55.695758 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:55.695699    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podStartSLOduration=1.695676293 podStartE2EDuration="1.695676293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:55.694566895 +0000 UTC m=+1178.714363522" watchObservedRunningTime="2026-09-18 18:58:55.695676293 +0000 UTC m=+1178.715472920"
Sep 18 18:58:56.510255 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:56.510214    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:56.515868 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:56.515840    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:56.680961 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:56.680843    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:58:56.681787 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:58:56.681766    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:59:00.642866 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.642840    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 18:59:00.763547 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.763513    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 18:59:00.767033 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.767014    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.771519 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.771492    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-880bee-dockercfg-ff72d\""
Sep 18 18:59:00.771876 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.771860    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-880bee\""
Sep 18 18:59:00.790557 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.790528    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 18:59:00.871417 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.871372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8gxn\" (UniqueName: \"kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.871588 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.871545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.972715 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.972608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.972715 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.972685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8gxn\" (UniqueName: \"kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.973427 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.973396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:00.994636 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:00.994601    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8gxn\" (UniqueName: \"kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn\") pod \"payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:01.077644 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.077606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:01.377431 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.377398    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 18:59:01.380784 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:59:01.380759    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fb1ac8_268d_4254_85e9_6a853986ad5c.slice/crio-3d401f9a7e226dc77b2cf9406d9e24e7c8dafb81d42b691ace7964bd545ac47a WatchSource:0}: Error finding container 3d401f9a7e226dc77b2cf9406d9e24e7c8dafb81d42b691ace7964bd545ac47a: Status 404 returned error can't find the container with id 3d401f9a7e226dc77b2cf9406d9e24e7c8dafb81d42b691ace7964bd545ac47a
Sep 18 18:59:01.708358 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.708275    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" event={"ID":"f7fb1ac8-268d-4254-85e9-6a853986ad5c","Type":"ContainerStarted","Data":"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"}
Sep 18 18:59:01.708358 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.708333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" event={"ID":"f7fb1ac8-268d-4254-85e9-6a853986ad5c","Type":"ContainerStarted","Data":"3d401f9a7e226dc77b2cf9406d9e24e7c8dafb81d42b691ace7964bd545ac47a"}
Sep 18 18:59:01.708757 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.708409    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:01.734786 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:01.734730    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" podStartSLOduration=1.7347136399999998 podStartE2EDuration="1.73471364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:01.732908208 +0000 UTC m=+1184.752704827" watchObservedRunningTime="2026-09-18 18:59:01.73471364 +0000 UTC m=+1184.754510301"
Sep 18 18:59:06.507033 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.506996    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 18:59:06.510630 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.510612    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.513723 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.513704    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-a6d509-serving-cert\""
Sep 18 18:59:06.528635 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.528608    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 18:59:06.633047 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.632997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.635732 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.635422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t787p\" (UniqueName: \"kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.635732 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.635532    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.635732 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.635558    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.737076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.737042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.737076 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.737077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.737348 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.737148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.737348 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.737165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t787p\" (UniqueName: \"kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.738012 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.737861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.739765 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.739743    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.739858 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.739784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.749748 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.749721    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t787p\" (UniqueName: \"kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p\") pod \"maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:06.821460 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:06.821429    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:07.101458 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.101434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 18:59:07.102748 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:59:07.102715    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34049c5_7a60_453c_a9f6_536c7754d315.slice/crio-8d8295a9984c0a85e4b5f4fcc57a7a3dcb00b0ef6a3bb63158b79dfaefc324e6 WatchSource:0}: Error finding container 8d8295a9984c0a85e4b5f4fcc57a7a3dcb00b0ef6a3bb63158b79dfaefc324e6: Status 404 returned error can't find the container with id 8d8295a9984c0a85e4b5f4fcc57a7a3dcb00b0ef6a3bb63158b79dfaefc324e6
Sep 18 18:59:07.550836 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.550804    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 18:59:07.554241 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.554225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.556841 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.556822    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-a6d509-dockercfg-k8ljh\""
Sep 18 18:59:07.557111 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.557098    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a6d509\""
Sep 18 18:59:07.574139 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.574110    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 18:59:07.644994 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.644920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.645214 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.645181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttxgg\" (UniqueName: \"kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.746096 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.746054    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttxgg\" (UniqueName: \"kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.746512 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.746485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.747571 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.747542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.752354 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.751385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" event={"ID":"f34049c5-7a60-453c-a9f6-536c7754d315","Type":"ContainerStarted","Data":"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"}
Sep 18 18:59:07.752354 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.751450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" event={"ID":"f34049c5-7a60-453c-a9f6-536c7754d315","Type":"ContainerStarted","Data":"8d8295a9984c0a85e4b5f4fcc57a7a3dcb00b0ef6a3bb63158b79dfaefc324e6"}
Sep 18 18:59:07.752354 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.751696    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:07.761182 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.761145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttxgg\" (UniqueName: \"kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg\") pod \"payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:07.783782 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.783729    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" podStartSLOduration=1.492716679 podStartE2EDuration="1.783713097s" totalImagesPullingTime="290.996418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:07.782574347 +0000 UTC m=+1190.802370973" watchObservedRunningTime="2026-09-18 18:59:07.783713097 +0000 UTC m=+1190.803509723"
Sep 18 18:59:07.865046 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:07.864995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:08.202682 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:08.202636    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 18:59:08.221085 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:59:08.221009    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a52c10_ee96_48ff_b81a_469b63028c5e.slice/crio-0fc68864cbda2f5f65b78cdd44056d32370c9282921f438ebacd3589ca4b3ccb WatchSource:0}: Error finding container 0fc68864cbda2f5f65b78cdd44056d32370c9282921f438ebacd3589ca4b3ccb: Status 404 returned error can't find the container with id 0fc68864cbda2f5f65b78cdd44056d32370c9282921f438ebacd3589ca4b3ccb
Sep 18 18:59:08.759618 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:08.759581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" event={"ID":"78a52c10-ee96-48ff-b81a-469b63028c5e","Type":"ContainerStarted","Data":"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"}
Sep 18 18:59:08.760003 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:08.759629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" event={"ID":"78a52c10-ee96-48ff-b81a-469b63028c5e","Type":"ContainerStarted","Data":"0fc68864cbda2f5f65b78cdd44056d32370c9282921f438ebacd3589ca4b3ccb"}
Sep 18 18:59:08.760003 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:08.759659    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:12.719385 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:12.719333    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 18:59:12.740724 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:12.740649    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" podStartSLOduration=5.740626728 podStartE2EDuration="5.740626728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:08.794133694 +0000 UTC m=+1191.813930321" watchObservedRunningTime="2026-09-18 18:59:12.740626728 +0000 UTC m=+1195.760423354"
Sep 18 18:59:13.765616 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:13.765539    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 18:59:15.651037 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:15.651000    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:59:15.651484 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:15.651058    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:59:16.807455 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.807421    2558 generic.go:356] "Generic (PLEG): container finished" podID="92ee097d-9630-400b-a66f-0dde1e81333e" containerID="59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2" exitCode=0
Sep 18 18:59:16.807912 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.807494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerDied","Data":"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"}
Sep 18 18:59:16.808029 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.808012    2558 scope.go:117] "RemoveContainer" containerID="59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"
Sep 18 18:59:16.809266 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809248    2558 generic.go:356] "Generic (PLEG): container finished" podID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerID="2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a" exitCode=0
Sep 18 18:59:16.809349 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809272    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:16.809349 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerDied","Data":"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"}
Sep 18 18:59:16.809349 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809342    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:16.809460 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809376    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:16.809606 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.809593    2558 scope.go:117] "RemoveContainer" containerID="2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"
Sep 18 18:59:16.810527 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.810503    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:16.810584 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.810559    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:16.810625 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:16.810588    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 18 18:59:17.820008 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:17.819944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerStarted","Data":"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"}
Sep 18 18:59:17.823799 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:17.823758    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerStarted","Data":"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"}
Sep 18 18:59:18.293249 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.293211    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:59:18.293458 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.293371    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:59:18.293458 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.293433    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 18:59:18.510588 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.510546    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:59:18.510787 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.510679    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 18:59:18.510787 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:18.510723    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 18:59:19.293839 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:19.293339    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:59:19.293839 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:19.293427    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 18:59:19.510292 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:19.510242    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 18:59:19.510473 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:19.510317    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 18:59:20.293854 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:20.293816    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:59:20.294481 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:20.294446    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 18:59:20.510634 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:20.510595    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 18:59:20.510929 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:20.510876    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 18:59:20.771071 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:20.770989    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 18:59:21.293024 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:21.292980    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:59:21.293236 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:21.293066    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 18:59:21.510262 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:21.510224    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 18:59:21.510851 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:21.510820    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 18:59:22.293397 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:22.293345    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:59:22.293621 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:22.293421    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 18:59:22.510246 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:22.510210    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 18:59:22.510516 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:22.510486    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 18:59:23.296902 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.296855    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:59:23.297367 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.297345    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:59:23.298197 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.298175    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 18:59:23.526672 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.526205    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:59:23.528868 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.527576    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:59:23.529511 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:23.529453    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 18:59:24.830065 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:24.830030    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 18:59:30.584283 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.584236    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 18:59:30.589841 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.589811    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.598682 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.598651    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-351045\""
Sep 18 18:59:30.599141 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.598716    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-351045-dockercfg-5mvzr\""
Sep 18 18:59:30.602970 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.602944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 18:59:30.692991 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.692956    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qncbb\" (UniqueName: \"kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.693165 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.693011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.794068 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.794028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qncbb\" (UniqueName: \"kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.794232 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.794087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.794718 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.794699    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.803488 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.803464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qncbb\" (UniqueName: \"kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb\") pod \"payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:30.913129 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:30.913040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:31.189836 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:31.189808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 18:59:31.192612 ip-10-0-132-229 kubenswrapper[2558]: W0918 18:59:31.192581    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb4b3de_ccef_45ed_a602_e055c3f91811.slice/crio-0b76a0a9c6866bc5cc56e92294a256a4d822ce79aa098d2650bc9403f55b14cf WatchSource:0}: Error finding container 0b76a0a9c6866bc5cc56e92294a256a4d822ce79aa098d2650bc9403f55b14cf: Status 404 returned error can't find the container with id 0b76a0a9c6866bc5cc56e92294a256a4d822ce79aa098d2650bc9403f55b14cf
Sep 18 18:59:31.907756 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:31.907716    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" event={"ID":"dbb4b3de-ccef-45ed-a602-e055c3f91811","Type":"ContainerStarted","Data":"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"}
Sep 18 18:59:31.908231 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:31.907771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" event={"ID":"dbb4b3de-ccef-45ed-a602-e055c3f91811","Type":"ContainerStarted","Data":"0b76a0a9c6866bc5cc56e92294a256a4d822ce79aa098d2650bc9403f55b14cf"}
Sep 18 18:59:31.908231 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:31.907835    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 18:59:31.923843 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:31.923787    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" podStartSLOduration=1.923767063 podStartE2EDuration="1.923767063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:31.922966294 +0000 UTC m=+1214.942762921" watchObservedRunningTime="2026-09-18 18:59:31.923767063 +0000 UTC m=+1214.943563691"
Sep 18 18:59:42.915383 ip-10-0-132-229 kubenswrapper[2558]: I0918 18:59:42.915351    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 19:00:00.166775 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.166734    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"]
Sep 18 19:00:00.169337 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.169315    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"
Sep 18 19:00:00.172271 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.172243    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xvbnc\""
Sep 18 19:00:00.179795 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.179615    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"
Sep 18 19:00:00.181632 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.181599    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"]
Sep 18 19:00:00.462724 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:00.462700    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"]
Sep 18 19:00:00.464618 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:00:00.464591    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d95074_cc6d_490d_8174_dd39c3823472.slice/crio-aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1 WatchSource:0}: Error finding container aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1: Status 404 returned error can't find the container with id aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1
Sep 18 19:00:01.058775 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:01.058735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55" event={"ID":"f9d95074-cc6d-490d-8174-dd39c3823472","Type":"ContainerStarted","Data":"aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1"}
Sep 18 19:00:05.615386 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:05.615354    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:00:10.043658 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.043562    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:00:10.047564 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.047541    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.050197 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.050176    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-25dc6358-serving-cert\""
Sep 18 19:00:10.060145 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.060118    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:00:10.176166 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.176125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.176334 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.176198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.176334 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.176300    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.176415 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.176354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28ncd\" (UniqueName: \"kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.277666 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.277630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.277840 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.277687    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ncd\" (UniqueName: \"kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.277840 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.277721    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.277840 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.277759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.278324 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.278304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.280436 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.280414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.280525 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.280454    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.286223 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.286205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ncd\" (UniqueName: \"kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd\") pod \"maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.362116 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.362039    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:10.637147 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:10.637099    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:00:10.642397 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:00:10.640140    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd152ae27_5a9c_41af_b06f_95b0826772e5.slice/crio-0908c25ea90a7023770a4e5d02094779e323b9f1757c00827d7d1e16bca06cd2 WatchSource:0}: Error finding container 0908c25ea90a7023770a4e5d02094779e323b9f1757c00827d7d1e16bca06cd2: Status 404 returned error can't find the container with id 0908c25ea90a7023770a4e5d02094779e323b9f1757c00827d7d1e16bca06cd2
Sep 18 19:00:11.116800 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:11.116772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" event={"ID":"d152ae27-5a9c-41af-b06f-95b0826772e5","Type":"ContainerStarted","Data":"0908c25ea90a7023770a4e5d02094779e323b9f1757c00827d7d1e16bca06cd2"}
Sep 18 19:00:12.124276 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:12.124217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" event={"ID":"d152ae27-5a9c-41af-b06f-95b0826772e5","Type":"ContainerStarted","Data":"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"}
Sep 18 19:00:12.124758 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:12.124331    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:12.151362 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:12.151300    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" podStartSLOduration=1.692748457 podStartE2EDuration="2.151281902s" totalImagesPullingTime="458.533445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:12.149066284 +0000 UTC m=+1255.168862910" watchObservedRunningTime="2026-09-18 19:00:12.151281902 +0000 UTC m=+1255.171078528"
Sep 18 19:00:13.130408 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:13.130364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55" event={"ID":"f9d95074-cc6d-490d-8174-dd39c3823472","Type":"ContainerStarted","Data":"c42c2b547b76fabba3c8bb969b11f7c0282aafc2c2101df892532346da84ff83"}
Sep 18 19:00:13.157255 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:13.157198    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55" podStartSLOduration=1.172365493 podStartE2EDuration="13.157181361s" totalImagesPullingTime="11.984815868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:13.15711306 +0000 UTC m=+1256.176909704" watchObservedRunningTime="2026-09-18 19:00:13.157181361 +0000 UTC m=+1256.176977986"
Sep 18 19:00:14.136884 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:14.136840    2558 generic.go:356] "Generic (PLEG): container finished" podID="f9d95074-cc6d-490d-8174-dd39c3823472" containerID="c42c2b547b76fabba3c8bb969b11f7c0282aafc2c2101df892532346da84ff83" exitCode=0
Sep 18 19:00:14.137317 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:14.136931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55" event={"ID":"f9d95074-cc6d-490d-8174-dd39c3823472","Type":"ContainerDied","Data":"c42c2b547b76fabba3c8bb969b11f7c0282aafc2c2101df892532346da84ff83"}
Sep 18 19:00:15.285103 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:15.285078    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"
Sep 18 19:00:16.148967 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:16.148941    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"
Sep 18 19:00:16.148967 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:16.148954    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55" event={"ID":"f9d95074-cc6d-490d-8174-dd39c3823472","Type":"ContainerDied","Data":"aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1"}
Sep 18 19:00:16.149185 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:16.149006    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aca95fc23abeb94d5b51f9ce87f0135ecd92b4963491026fdf7c378c13a316c1"
Sep 18 19:00:17.753110 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:17.753068    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 19:00:18.136531 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:18.136501    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:00:29.067536 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.067502    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 19:00:29.067954 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.067718    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" podUID="b3e2ad0e-fd03-43f6-948b-929fce96c467" containerName="maas-api" containerID="cri-o://0f02ed344bc139483abbf8f427543b4b44d9044aeddf68423fb469eb9be21cab" gracePeriod=30
Sep 18 19:00:29.215019 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.214981    2558 generic.go:356] "Generic (PLEG): container finished" podID="b3e2ad0e-fd03-43f6-948b-929fce96c467" containerID="0f02ed344bc139483abbf8f427543b4b44d9044aeddf68423fb469eb9be21cab" exitCode=0
Sep 18 19:00:29.215019 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.215003    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" event={"ID":"b3e2ad0e-fd03-43f6-948b-929fce96c467","Type":"ContainerDied","Data":"0f02ed344bc139483abbf8f427543b4b44d9044aeddf68423fb469eb9be21cab"}
Sep 18 19:00:29.320668 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.320610    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 19:00:29.368156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.367945    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") pod \"b3e2ad0e-fd03-43f6-948b-929fce96c467\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") "
Sep 18 19:00:29.368156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.368015    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca\") pod \"b3e2ad0e-fd03-43f6-948b-929fce96c467\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") "
Sep 18 19:00:29.368156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.368051    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs\") pod \"b3e2ad0e-fd03-43f6-948b-929fce96c467\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") "
Sep 18 19:00:29.368156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.368101    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn75j\" (UniqueName: \"kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j\") pod \"b3e2ad0e-fd03-43f6-948b-929fce96c467\" (UID: \"b3e2ad0e-fd03-43f6-948b-929fce96c467\") "
Sep 18 19:00:29.368743 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.368501    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3e2ad0e-fd03-43f6-948b-929fce96c467" (UID: "b3e2ad0e-fd03-43f6-948b-929fce96c467"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:29.369513 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.369481    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e2ad0e-fd03-43f6-948b-929fce96c467-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.374056 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.374005    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3e2ad0e-fd03-43f6-948b-929fce96c467" (UID: "b3e2ad0e-fd03-43f6-948b-929fce96c467"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:29.374183 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.374069    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j" (OuterVolumeSpecName: "kube-api-access-pn75j") pod "b3e2ad0e-fd03-43f6-948b-929fce96c467" (UID: "b3e2ad0e-fd03-43f6-948b-929fce96c467"). InnerVolumeSpecName "kube-api-access-pn75j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:29.375299 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.375270    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b3e2ad0e-fd03-43f6-948b-929fce96c467" (UID: "b3e2ad0e-fd03-43f6-948b-929fce96c467"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:29.471084 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.471050    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn75j\" (UniqueName: \"kubernetes.io/projected/b3e2ad0e-fd03-43f6-948b-929fce96c467-kube-api-access-pn75j\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.471084 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.471078    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.471084 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:29.471087    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3e2ad0e-fd03-43f6-948b-929fce96c467-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:30.221118 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:30.221088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx" event={"ID":"b3e2ad0e-fd03-43f6-948b-929fce96c467","Type":"ContainerDied","Data":"327e9eb912d299eb6a3a3eb2f0082007074f6506a33fa9bde75cd2849d827593"}
Sep 18 19:00:30.221537 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:30.221148    2558 scope.go:117] "RemoveContainer" containerID="0f02ed344bc139483abbf8f427543b4b44d9044aeddf68423fb469eb9be21cab"
Sep 18 19:00:30.221537 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:30.221101    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"
Sep 18 19:00:30.252415 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:30.252381    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 19:00:30.266263 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:30.266237    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9d9f36-657f48b9b6-d2btx"]
Sep 18 19:00:31.434250 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:31.434211    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3e2ad0e-fd03-43f6-948b-929fce96c467" path="/var/lib/kubelet/pods/b3e2ad0e-fd03-43f6-948b-929fce96c467/volumes"
Sep 18 19:00:32.777886 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:32.777839    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" podUID="9cd64820-7840-47a7-9a63-3d817e153754" containerName="main" containerID="cri-o://a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167" gracePeriod=45
Sep 18 19:00:33.030778 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.030683    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 19:00:33.105494 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105462    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.105674 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105511    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7qss\" (UniqueName: \"kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.105674 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105537    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.105674 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105641    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.105837 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105691    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.105837 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache" (OuterVolumeSpecName: "model-cache") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.105837 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.105790    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home\") pod \"9cd64820-7840-47a7-9a63-3d817e153754\" (UID: \"9cd64820-7840-47a7-9a63-3d817e153754\") "
Sep 18 19:00:33.106146 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.106120    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home" (OuterVolumeSpecName: "home") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.106212 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.106188    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-model-cache\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.106268 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.106223    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.108039 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.108015    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm" (OuterVolumeSpecName: "dshm") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.108261 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.108241    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:33.108392 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.108257    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss" (OuterVolumeSpecName: "kube-api-access-t7qss") pod "9cd64820-7840-47a7-9a63-3d817e153754" (UID: "9cd64820-7840-47a7-9a63-3d817e153754"). InnerVolumeSpecName "kube-api-access-t7qss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:33.206997 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.206959    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7qss\" (UniqueName: \"kubernetes.io/projected/9cd64820-7840-47a7-9a63-3d817e153754-kube-api-access-t7qss\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.206997 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.206995    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd64820-7840-47a7-9a63-3d817e153754-tls-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.207225 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.207025    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-tmp-dir\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.207225 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.207033    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-dshm\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.207225 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.207040    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd64820-7840-47a7-9a63-3d817e153754-home\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.239516 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.239478    2558 generic.go:356] "Generic (PLEG): container finished" podID="9cd64820-7840-47a7-9a63-3d817e153754" containerID="a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167" exitCode=0
Sep 18 19:00:33.239516 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.239519    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"
Sep 18 19:00:33.239721 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.239560    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" event={"ID":"9cd64820-7840-47a7-9a63-3d817e153754","Type":"ContainerDied","Data":"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"}
Sep 18 19:00:33.239721 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.239606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj" event={"ID":"9cd64820-7840-47a7-9a63-3d817e153754","Type":"ContainerDied","Data":"24093912389bebe8037ef4fede86ca0894f24124b90455d5a23600ce06607e46"}
Sep 18 19:00:33.239721 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.239625    2558 scope.go:117] "RemoveContainer" containerID="a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"
Sep 18 19:00:33.248950 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.248932    2558 scope.go:117] "RemoveContainer" containerID="a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"
Sep 18 19:00:33.249270 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:00:33.249249    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167\": container with ID starting with a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167 not found: ID does not exist" containerID="a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"
Sep 18 19:00:33.249360 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.249276    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167"} err="failed to get container status \"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167\": rpc error: code = NotFound desc = could not find container \"a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167\": container with ID starting with a555df72b5f852b0ea88cd9d1d360678c207996df40fe31a9fc9bdf1dabfc167 not found: ID does not exist"
Sep 18 19:00:33.270169 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.270142    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 19:00:33.274466 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.274444    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-2-w3-9d9f36-kserve-74b4ccf88-6xssj"]
Sep 18 19:00:33.435605 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:33.435316    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd64820-7840-47a7-9a63-3d817e153754" path="/var/lib/kubelet/pods/9cd64820-7840-47a7-9a63-3d817e153754/volumes"
Sep 18 19:00:40.025925 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:00:40.025857    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:01:07.834449 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:01:07.834411    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:01:39.144642 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:01:39.144556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:02:00.693301 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:00.693252    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:02:00.693770 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:00.693556    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" podUID="d152ae27-5a9c-41af-b06f-95b0826772e5" containerName="maas-api" containerID="cri-o://9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd" gracePeriod=30
Sep 18 19:02:00.950646 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:00.950585    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:02:01.031349 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.031313    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28ncd\" (UniqueName: \"kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd\") pod \"d152ae27-5a9c-41af-b06f-95b0826772e5\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") "
Sep 18 19:02:01.031349 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.031352    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca\") pod \"d152ae27-5a9c-41af-b06f-95b0826772e5\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") "
Sep 18 19:02:01.031551 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.031393    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls\") pod \"d152ae27-5a9c-41af-b06f-95b0826772e5\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") "
Sep 18 19:02:01.031551 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.031449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs\") pod \"d152ae27-5a9c-41af-b06f-95b0826772e5\" (UID: \"d152ae27-5a9c-41af-b06f-95b0826772e5\") "
Sep 18 19:02:01.031835 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.031798    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d152ae27-5a9c-41af-b06f-95b0826772e5" (UID: "d152ae27-5a9c-41af-b06f-95b0826772e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:01.033669 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.033629    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d152ae27-5a9c-41af-b06f-95b0826772e5" (UID: "d152ae27-5a9c-41af-b06f-95b0826772e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:01.033777 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.033670    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d152ae27-5a9c-41af-b06f-95b0826772e5" (UID: "d152ae27-5a9c-41af-b06f-95b0826772e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:01.033777 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.033707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd" (OuterVolumeSpecName: "kube-api-access-28ncd") pod "d152ae27-5a9c-41af-b06f-95b0826772e5" (UID: "d152ae27-5a9c-41af-b06f-95b0826772e5"). InnerVolumeSpecName "kube-api-access-28ncd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:01.132319 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.132284    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28ncd\" (UniqueName: \"kubernetes.io/projected/d152ae27-5a9c-41af-b06f-95b0826772e5-kube-api-access-28ncd\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.132319 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.132316    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d152ae27-5a9c-41af-b06f-95b0826772e5-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.132319 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.132327    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.132559 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.132335    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d152ae27-5a9c-41af-b06f-95b0826772e5-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.673566 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.673527    2558 generic.go:356] "Generic (PLEG): container finished" podID="d152ae27-5a9c-41af-b06f-95b0826772e5" containerID="9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd" exitCode=0
Sep 18 19:02:01.673566 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.673572    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"
Sep 18 19:02:01.673789 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.673616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" event={"ID":"d152ae27-5a9c-41af-b06f-95b0826772e5","Type":"ContainerDied","Data":"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"}
Sep 18 19:02:01.673789 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.673666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb" event={"ID":"d152ae27-5a9c-41af-b06f-95b0826772e5","Type":"ContainerDied","Data":"0908c25ea90a7023770a4e5d02094779e323b9f1757c00827d7d1e16bca06cd2"}
Sep 18 19:02:01.673789 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.673690    2558 scope.go:117] "RemoveContainer" containerID="9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"
Sep 18 19:02:01.682448 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.682431    2558 scope.go:117] "RemoveContainer" containerID="9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"
Sep 18 19:02:01.682714 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:02:01.682695    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd\": container with ID starting with 9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd not found: ID does not exist" containerID="9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"
Sep 18 19:02:01.682770 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.682723    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd"} err="failed to get container status \"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd\": rpc error: code = NotFound desc = could not find container \"9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd\": container with ID starting with 9a565dec9b7a3850dbc96f14efe53acab69a69f3d9e757ac8ad59c5c095f8ebd not found: ID does not exist"
Sep 18 19:02:01.690493 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.690468    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:02:01.696838 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:01.696772    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25dc6358-64f58dd4b5-8xnlb"]
Sep 18 19:02:03.435590 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:03.435556    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d152ae27-5a9c-41af-b06f-95b0826772e5" path="/var/lib/kubelet/pods/d152ae27-5a9c-41af-b06f-95b0826772e5/volumes"
Sep 18 19:02:07.809764 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:07.809729    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:02:19.853019 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:19.852985    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 19:02:34.872770 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:34.872724    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" podUID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" containerName="main" containerID="cri-o://89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754" gracePeriod=45
Sep 18 19:02:35.121217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.121194    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 19:02:35.175928 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.175825    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.175928 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.175870    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.176145 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.175935    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.176145 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.175979    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb5vf\" (UniqueName: \"kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.176145 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.176145 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176062    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir\") pod \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\" (UID: \"dddbc95a-b594-4ecf-a974-19b7e0afabf4\") "
Sep 18 19:02:35.176350 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176181    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache" (OuterVolumeSpecName: "model-cache") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:35.176350 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176279    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home" (OuterVolumeSpecName: "home") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:35.176453 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176385    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:35.176506 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176482    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-home\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.176506 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176498    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tmp-dir\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.176591 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.176512    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-model-cache\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.178368 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.178342    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf" (OuterVolumeSpecName: "kube-api-access-sb5vf") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "kube-api-access-sb5vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:35.178554 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.178538    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:35.178600 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.178569    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm" (OuterVolumeSpecName: "dshm") pod "dddbc95a-b594-4ecf-a974-19b7e0afabf4" (UID: "dddbc95a-b594-4ecf-a974-19b7e0afabf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:35.277043 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.277005    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddbc95a-b594-4ecf-a974-19b7e0afabf4-tls-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.277043 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.277037    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb5vf\" (UniqueName: \"kubernetes.io/projected/dddbc95a-b594-4ecf-a974-19b7e0afabf4-kube-api-access-sb5vf\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.277043 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.277047    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dddbc95a-b594-4ecf-a974-19b7e0afabf4-dshm\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:02:35.801401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.801372    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:02:35.844033 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.843999    2558 generic.go:356] "Generic (PLEG): container finished" podID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" containerID="89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754" exitCode=0
Sep 18 19:02:35.844207 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.844041    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"
Sep 18 19:02:35.844207 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.844078    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" event={"ID":"dddbc95a-b594-4ecf-a974-19b7e0afabf4","Type":"ContainerDied","Data":"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"}
Sep 18 19:02:35.844207 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.844120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb" event={"ID":"dddbc95a-b594-4ecf-a974-19b7e0afabf4","Type":"ContainerDied","Data":"2f11d9e48f6917e1cf816f8c261fb2f65219b7e041b53a479bed17e661df6445"}
Sep 18 19:02:35.844207 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.844137    2558 scope.go:117] "RemoveContainer" containerID="89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"
Sep 18 19:02:35.852776 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.852758    2558 scope.go:117] "RemoveContainer" containerID="89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"
Sep 18 19:02:35.853081 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:02:35.853063    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754\": container with ID starting with 89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754 not found: ID does not exist" containerID="89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"
Sep 18 19:02:35.853148 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.853089    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754"} err="failed to get container status \"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754\": rpc error: code = NotFound desc = could not find container \"89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754\": container with ID starting with 89d1a8b9ba356fe6e4a07c3a1ca672f594bb127377788862b99a5bcadebe5754 not found: ID does not exist"
Sep 18 19:02:35.859293 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.859272    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 19:02:35.862756 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:35.862739    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc289c/distinct-w0-cc289c-kserve-58c8bdd78c-lbndb"]
Sep 18 19:02:37.437972 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:37.437916    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" path="/var/lib/kubelet/pods/dddbc95a-b594-4ecf-a974-19b7e0afabf4/volumes"
Sep 18 19:02:56.604484 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:02:56.604393    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:03:12.205347 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:03:12.205262    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:03:28.400841 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:03:28.400798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:03:43.719107 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:03:43.718951    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:03:59.311583 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:03:59.311545    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:04:14.718971 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:14.718929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:04:28.680030 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.679994    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680384    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d152ae27-5a9c-41af-b06f-95b0826772e5" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680399    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d152ae27-5a9c-41af-b06f-95b0826772e5" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680417    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3e2ad0e-fd03-43f6-948b-929fce96c467" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680423    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3e2ad0e-fd03-43f6-948b-929fce96c467" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680430    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d95074-cc6d-490d-8174-dd39c3823472" containerName="cleanup"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680436    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d95074-cc6d-490d-8174-dd39c3823472" containerName="cleanup"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680442    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd64820-7840-47a7-9a63-3d817e153754" containerName="main"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680447    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd64820-7840-47a7-9a63-3d817e153754" containerName="main"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680460    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" containerName="main"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680468    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" containerName="main"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680524    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3e2ad0e-fd03-43f6-948b-929fce96c467" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680530    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d95074-cc6d-490d-8174-dd39c3823472" containerName="cleanup"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680537    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd64820-7840-47a7-9a63-3d817e153754" containerName="main"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680544    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d152ae27-5a9c-41af-b06f-95b0826772e5" containerName="maas-api"
Sep 18 19:04:28.682375 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.680551    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dddbc95a-b594-4ecf-a974-19b7e0afabf4" containerName="main"
Sep 18 19:04:28.683299 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.683282    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.685789 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.685768    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-02e2c9\""
Sep 18 19:04:28.685961 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.685943    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-02e2c9-dockercfg-thbr9\""
Sep 18 19:04:28.695105 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.695078    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:04:28.709791 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.709755    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:04:28.713156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.713129    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.722457 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.722425    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:04:28.734302 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.734270    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv472\" (UniqueName: \"kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.734302 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.734318    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.734523 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.734446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.734523 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.734503    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rfq\" (UniqueName: \"kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.835079 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.835037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rfq\" (UniqueName: \"kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.835079 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.835086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv472\" (UniqueName: \"kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.835357 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.835121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.835357 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.835176    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.836129 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.836106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.836191 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.836106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.843392 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.843368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv472\" (UniqueName: \"kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472\") pod \"payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") " pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:28.843516 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.843430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rfq\" (UniqueName: \"kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq\") pod \"payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:28.994193 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:28.994104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:29.024599 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.024564    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:29.268143 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.268028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:04:29.271664 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:04:29.271622    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f653df1_fbbf_4471_8bb0_e219064fa13b.slice/crio-285d7329e8968fb00784038a89c3671b2a0537b7d7d3c4612f89a6b9a464d4be WatchSource:0}: Error finding container 285d7329e8968fb00784038a89c3671b2a0537b7d7d3c4612f89a6b9a464d4be: Status 404 returned error can't find the container with id 285d7329e8968fb00784038a89c3671b2a0537b7d7d3c4612f89a6b9a464d4be
Sep 18 19:04:29.304656 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.304630    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:04:29.307050 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:04:29.307020    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bbd206_5489_4521_a0dd_f871c5a2d6d1.slice/crio-91697a11dd556146b61383120e838ff024867726d966c9226a5b28022e029029 WatchSource:0}: Error finding container 91697a11dd556146b61383120e838ff024867726d966c9226a5b28022e029029: Status 404 returned error can't find the container with id 91697a11dd556146b61383120e838ff024867726d966c9226a5b28022e029029
Sep 18 19:04:29.408966 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.408929    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" event={"ID":"d0bbd206-5489-4521-a0dd-f871c5a2d6d1","Type":"ContainerStarted","Data":"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"}
Sep 18 19:04:29.409169 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.408988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" event={"ID":"d0bbd206-5489-4521-a0dd-f871c5a2d6d1","Type":"ContainerStarted","Data":"91697a11dd556146b61383120e838ff024867726d966c9226a5b28022e029029"}
Sep 18 19:04:29.409169 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.409034    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:29.411041 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.410996    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" event={"ID":"7f653df1-fbbf-4471-8bb0-e219064fa13b","Type":"ContainerStarted","Data":"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"}
Sep 18 19:04:29.411221 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.411064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" event={"ID":"7f653df1-fbbf-4471-8bb0-e219064fa13b","Type":"ContainerStarted","Data":"285d7329e8968fb00784038a89c3671b2a0537b7d7d3c4612f89a6b9a464d4be"}
Sep 18 19:04:29.411221 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.411173    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:29.428722 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.428650    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" podStartSLOduration=1.428629508 podStartE2EDuration="1.428629508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:29.427615652 +0000 UTC m=+1512.447412283" watchObservedRunningTime="2026-09-18 19:04:29.428629508 +0000 UTC m=+1512.448426136"
Sep 18 19:04:29.445862 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:29.445810    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" podStartSLOduration=1.4457900989999999 podStartE2EDuration="1.445790099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:29.443866204 +0000 UTC m=+1512.463662830" watchObservedRunningTime="2026-09-18 19:04:29.445790099 +0000 UTC m=+1512.465586725"
Sep 18 19:04:34.411690 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:34.411649    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:04:40.418114 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:40.418033    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:04:40.418574 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:40.418145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:04:56.012370 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:04:56.010300    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:05:12.209384 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:12.209339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:05:32.207192 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:32.207155    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:05:35.730639 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:35.730605    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:05:35.731153 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:35.730926    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" podUID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" containerName="payload-processing" containerID="cri-o://a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46" gracePeriod=30
Sep 18 19:05:35.740674 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:35.740639    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:05:35.740985 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:35.740951    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" podUID="7f653df1-fbbf-4471-8bb0-e219064fa13b" containerName="payload-pre-processing" containerID="cri-o://bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88" gracePeriod=30
Sep 18 19:05:35.992777 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:35.992706    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:05:36.026568 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.026538    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:05:36.057687 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.056949    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv472\" (UniqueName: \"kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472\") pod \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") "
Sep 18 19:05:36.057687 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.057109    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume\") pod \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\" (UID: \"d0bbd206-5489-4521-a0dd-f871c5a2d6d1\") "
Sep 18 19:05:36.059760 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.059721    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0bbd206-5489-4521-a0dd-f871c5a2d6d1" (UID: "d0bbd206-5489-4521-a0dd-f871c5a2d6d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:36.061858 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.061827    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472" (OuterVolumeSpecName: "kube-api-access-vv472") pod "d0bbd206-5489-4521-a0dd-f871c5a2d6d1" (UID: "d0bbd206-5489-4521-a0dd-f871c5a2d6d1"). InnerVolumeSpecName "kube-api-access-vv472". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:36.160808 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.160768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume\") pod \"7f653df1-fbbf-4471-8bb0-e219064fa13b\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") "
Sep 18 19:05:36.161043 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.160921    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5rfq\" (UniqueName: \"kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq\") pod \"7f653df1-fbbf-4471-8bb0-e219064fa13b\" (UID: \"7f653df1-fbbf-4471-8bb0-e219064fa13b\") "
Sep 18 19:05:36.161134 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.161114    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv472\" (UniqueName: \"kubernetes.io/projected/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-kube-api-access-vv472\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.161134 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.161129    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0bbd206-5489-4521-a0dd-f871c5a2d6d1-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.161272 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.161242    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f653df1-fbbf-4471-8bb0-e219064fa13b" (UID: "7f653df1-fbbf-4471-8bb0-e219064fa13b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:36.163148 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.163125    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq" (OuterVolumeSpecName: "kube-api-access-p5rfq") pod "7f653df1-fbbf-4471-8bb0-e219064fa13b" (UID: "7f653df1-fbbf-4471-8bb0-e219064fa13b"). InnerVolumeSpecName "kube-api-access-p5rfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:36.262575 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.262489    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f653df1-fbbf-4471-8bb0-e219064fa13b-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.262575 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.262516    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5rfq\" (UniqueName: \"kubernetes.io/projected/7f653df1-fbbf-4471-8bb0-e219064fa13b-kube-api-access-p5rfq\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.750384 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.750354    2558 generic.go:356] "Generic (PLEG): container finished" podID="7f653df1-fbbf-4471-8bb0-e219064fa13b" containerID="bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88" exitCode=0
Sep 18 19:05:36.750805 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.750405    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"
Sep 18 19:05:36.750805 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.750431    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" event={"ID":"7f653df1-fbbf-4471-8bb0-e219064fa13b","Type":"ContainerDied","Data":"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"}
Sep 18 19:05:36.750805 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.750499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf" event={"ID":"7f653df1-fbbf-4471-8bb0-e219064fa13b","Type":"ContainerDied","Data":"285d7329e8968fb00784038a89c3671b2a0537b7d7d3c4612f89a6b9a464d4be"}
Sep 18 19:05:36.750805 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.750533    2558 scope.go:117] "RemoveContainer" containerID="bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"
Sep 18 19:05:36.752007 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.751987    2558 generic.go:356] "Generic (PLEG): container finished" podID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" containerID="a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46" exitCode=0
Sep 18 19:05:36.752086 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.752036    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"
Sep 18 19:05:36.752086 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.752076    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" event={"ID":"d0bbd206-5489-4521-a0dd-f871c5a2d6d1","Type":"ContainerDied","Data":"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"}
Sep 18 19:05:36.752204 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.752125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj" event={"ID":"d0bbd206-5489-4521-a0dd-f871c5a2d6d1","Type":"ContainerDied","Data":"91697a11dd556146b61383120e838ff024867726d966c9226a5b28022e029029"}
Sep 18 19:05:36.761379 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.761361    2558 scope.go:117] "RemoveContainer" containerID="bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"
Sep 18 19:05:36.761675 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:05:36.761656    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88\": container with ID starting with bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88 not found: ID does not exist" containerID="bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"
Sep 18 19:05:36.761777 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.761684    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88"} err="failed to get container status \"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88\": rpc error: code = NotFound desc = could not find container \"bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88\": container with ID starting with bf34f4ad82c979cc11cf841fd4ccd98ef24a9e2570a0cd8b5239b642da1d4d88 not found: ID does not exist"
Sep 18 19:05:36.761777 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.761702    2558 scope.go:117] "RemoveContainer" containerID="a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"
Sep 18 19:05:36.770725 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.770707    2558 scope.go:117] "RemoveContainer" containerID="a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"
Sep 18 19:05:36.771087 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:05:36.771067    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46\": container with ID starting with a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46 not found: ID does not exist" containerID="a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"
Sep 18 19:05:36.771153 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.771094    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46"} err="failed to get container status \"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46\": rpc error: code = NotFound desc = could not find container \"a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46\": container with ID starting with a7d55ba94a602913bff84268a811ab13460edb8cb4cd79c9964e5750c69abb46 not found: ID does not exist"
Sep 18 19:05:36.776026 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.776002    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:05:36.778684 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.778661    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02e2c9-dc4454bcd-kbklf"]
Sep 18 19:05:36.788603 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.788575    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:05:36.793303 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:36.793279    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02e2c9-b4c4d5bc5-qczqj"]
Sep 18 19:05:37.434633 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:37.434602    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f653df1-fbbf-4471-8bb0-e219064fa13b" path="/var/lib/kubelet/pods/7f653df1-fbbf-4471-8bb0-e219064fa13b/volumes"
Sep 18 19:05:37.435039 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:37.435023    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" path="/var/lib/kubelet/pods/d0bbd206-5489-4521-a0dd-f871c5a2d6d1/volumes"
Sep 18 19:05:46.508767 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:46.508725    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:05:53.438212 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.438176    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 19:05:53.438978 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.438925    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" podUID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" containerName="payload-pre-processing" containerID="cri-o://9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d" gracePeriod=30
Sep 18 19:05:53.704351 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.704264    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 19:05:53.819499 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.819465    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8gxn\" (UniqueName: \"kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn\") pod \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") "
Sep 18 19:05:53.819685 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.819570    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume\") pod \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\" (UID: \"f7fb1ac8-268d-4254-85e9-6a853986ad5c\") "
Sep 18 19:05:53.820284 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.820185    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7fb1ac8-268d-4254-85e9-6a853986ad5c" (UID: "f7fb1ac8-268d-4254-85e9-6a853986ad5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:53.822229 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.822201    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn" (OuterVolumeSpecName: "kube-api-access-f8gxn") pod "f7fb1ac8-268d-4254-85e9-6a853986ad5c" (UID: "f7fb1ac8-268d-4254-85e9-6a853986ad5c"). InnerVolumeSpecName "kube-api-access-f8gxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:53.838586 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.838553    2558 generic.go:356] "Generic (PLEG): container finished" podID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" containerID="9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d" exitCode=0
Sep 18 19:05:53.838700 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.838600    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"
Sep 18 19:05:53.838700 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.838615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" event={"ID":"f7fb1ac8-268d-4254-85e9-6a853986ad5c","Type":"ContainerDied","Data":"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"}
Sep 18 19:05:53.838700 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.838665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr" event={"ID":"f7fb1ac8-268d-4254-85e9-6a853986ad5c","Type":"ContainerDied","Data":"3d401f9a7e226dc77b2cf9406d9e24e7c8dafb81d42b691ace7964bd545ac47a"}
Sep 18 19:05:53.838700 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.838687    2558 scope.go:117] "RemoveContainer" containerID="9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"
Sep 18 19:05:53.847940 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.847922    2558 scope.go:117] "RemoveContainer" containerID="9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"
Sep 18 19:05:53.848217 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:05:53.848199    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d\": container with ID starting with 9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d not found: ID does not exist" containerID="9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"
Sep 18 19:05:53.848284 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.848224    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d"} err="failed to get container status \"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d\": rpc error: code = NotFound desc = could not find container \"9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d\": container with ID starting with 9ef0359cda236d9cd4adaf782ee3ff693783b075309cff3dec99e9623002d69d not found: ID does not exist"
Sep 18 19:05:53.859292 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.859263    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 19:05:53.862781 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.862763    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-880bee-6656c4d966-mfxnr"]
Sep 18 19:05:53.920676 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.920638    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fb1ac8-268d-4254-85e9-6a853986ad5c-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.920676 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:53.920672    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8gxn\" (UniqueName: \"kubernetes.io/projected/f7fb1ac8-268d-4254-85e9-6a853986ad5c-kube-api-access-f8gxn\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:05:55.440396 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:55.440354    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" path="/var/lib/kubelet/pods/f7fb1ac8-268d-4254-85e9-6a853986ad5c/volumes"
Sep 18 19:05:59.965321 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:59.965287    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 19:05:59.965734 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:05:59.965566    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy" containerID="cri-o://17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6" gracePeriod=30
Sep 18 19:06:01.002373 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:01.002340    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:06:05.216320 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.216295    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 19:06:05.320379 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320333    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320379 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320393    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320410    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320431    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320471    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320502    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320649    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320670    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320702    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vcj7\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7\") pod \"853c9fcd-c69a-40d4-aeab-2f7113077e37\" (UID: \"853c9fcd-c69a-40d4-aeab-2f7113077e37\") "
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320739    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320832    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data" (OuterVolumeSpecName: "istio-data") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:05.320960 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.320940    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:05.321217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.321066    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.321217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.321088    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c9fcd-c69a-40d4-aeab-2f7113077e37-istiod-ca-cert\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.321217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.321086    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:05.321217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.321102    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-data\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.321217 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.321115    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-workload-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.322971 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.322950    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7" (OuterVolumeSpecName: "kube-api-access-7vcj7") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "kube-api-access-7vcj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:05.323040 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.322989    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:06:05.323412 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.323386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token" (OuterVolumeSpecName: "istio-token") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:05.323412 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.323402    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "853c9fcd-c69a-40d4-aeab-2f7113077e37" (UID: "853c9fcd-c69a-40d4-aeab-2f7113077e37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:05.422321 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.422283    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.422321 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.422315    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-credential-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.422321 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.422324    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-podinfo\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.422321 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.422332    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c9fcd-c69a-40d4-aeab-2f7113077e37-istio-envoy\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.422580 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.422341    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vcj7\" (UniqueName: \"kubernetes.io/projected/853c9fcd-c69a-40d4-aeab-2f7113077e37-kube-api-access-7vcj7\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:05.903216 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.903176    2558 generic.go:356] "Generic (PLEG): container finished" podID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerID="17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6" exitCode=0
Sep 18 19:06:05.903395 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.903227    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"
Sep 18 19:06:05.903395 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.903259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerDied","Data":"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"}
Sep 18 19:06:05.903395 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.903325    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5" event={"ID":"853c9fcd-c69a-40d4-aeab-2f7113077e37","Type":"ContainerDied","Data":"5e616a1b2fbce994957a6c65a76bab4ef77e6507c82523b26915284166dfd557"}
Sep 18 19:06:05.903395 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.903349    2558 scope.go:117] "RemoveContainer" containerID="17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"
Sep 18 19:06:05.912212 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.912193    2558 scope.go:117] "RemoveContainer" containerID="2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"
Sep 18 19:06:05.920251 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.920234    2558 scope.go:117] "RemoveContainer" containerID="17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"
Sep 18 19:06:05.920544 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:06:05.920523    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6\": container with ID starting with 17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6 not found: ID does not exist" containerID="17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"
Sep 18 19:06:05.920618 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.920555    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6"} err="failed to get container status \"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6\": rpc error: code = NotFound desc = could not find container \"17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6\": container with ID starting with 17f9645af4204c05a9dea4c7769a867b5033341f02ca33f9bcda71fbaa6cd3c6 not found: ID does not exist"
Sep 18 19:06:05.920618 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.920573    2558 scope.go:117] "RemoveContainer" containerID="2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"
Sep 18 19:06:05.920836 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:06:05.920817    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a\": container with ID starting with 2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a not found: ID does not exist" containerID="2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"
Sep 18 19:06:05.920919 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.920841    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a"} err="failed to get container status \"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a\": rpc error: code = NotFound desc = could not find container \"2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a\": container with ID starting with 2c093a9a5baaa4f5998295252db78a8846efaf939810cb550c294f064598cb7a not found: ID does not exist"
Sep 18 19:06:05.921484 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.921465    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 19:06:05.927336 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:05.927314    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-880bee-openshift-default-6db5cc6788-w94c5"]
Sep 18 19:06:07.434649 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:07.434616    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" path="/var/lib/kubelet/pods/853c9fcd-c69a-40d4-aeab-2f7113077e37/volumes"
Sep 18 19:06:10.612919 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:10.612816    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:06:10.824685 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:10.824651    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 19:06:10.825177 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:10.825145    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" podUID="dbb4b3de-ccef-45ed-a602-e055c3f91811" containerName="payload-processing" containerID="cri-o://473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99" gracePeriod=30
Sep 18 19:06:11.090663 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.090629    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 19:06:11.177215 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.177179    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qncbb\" (UniqueName: \"kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb\") pod \"dbb4b3de-ccef-45ed-a602-e055c3f91811\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") "
Sep 18 19:06:11.177382 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.177269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume\") pod \"dbb4b3de-ccef-45ed-a602-e055c3f91811\" (UID: \"dbb4b3de-ccef-45ed-a602-e055c3f91811\") "
Sep 18 19:06:11.177640 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.177612    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbb4b3de-ccef-45ed-a602-e055c3f91811" (UID: "dbb4b3de-ccef-45ed-a602-e055c3f91811"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:11.179470 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.179448    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb" (OuterVolumeSpecName: "kube-api-access-qncbb") pod "dbb4b3de-ccef-45ed-a602-e055c3f91811" (UID: "dbb4b3de-ccef-45ed-a602-e055c3f91811"). InnerVolumeSpecName "kube-api-access-qncbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:11.278365 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.278338    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qncbb\" (UniqueName: \"kubernetes.io/projected/dbb4b3de-ccef-45ed-a602-e055c3f91811-kube-api-access-qncbb\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.278365 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.278365    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4b3de-ccef-45ed-a602-e055c3f91811-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.933139 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.933101    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb4b3de-ccef-45ed-a602-e055c3f91811" containerID="473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99" exitCode=0
Sep 18 19:06:11.933528 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.933147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"
Sep 18 19:06:11.933528 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.933198    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" event={"ID":"dbb4b3de-ccef-45ed-a602-e055c3f91811","Type":"ContainerDied","Data":"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"}
Sep 18 19:06:11.933528 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.933251    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx" event={"ID":"dbb4b3de-ccef-45ed-a602-e055c3f91811","Type":"ContainerDied","Data":"0b76a0a9c6866bc5cc56e92294a256a4d822ce79aa098d2650bc9403f55b14cf"}
Sep 18 19:06:11.933528 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.933273    2558 scope.go:117] "RemoveContainer" containerID="473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"
Sep 18 19:06:11.941921 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.941902    2558 scope.go:117] "RemoveContainer" containerID="473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"
Sep 18 19:06:11.942197 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:06:11.942173    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99\": container with ID starting with 473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99 not found: ID does not exist" containerID="473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"
Sep 18 19:06:11.942250 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.942206    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99"} err="failed to get container status \"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99\": rpc error: code = NotFound desc = could not find container \"473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99\": container with ID starting with 473a5d991cf788aabb6cce92b67296bb7ba3663e80edde82f7d4e59291929c99 not found: ID does not exist"
Sep 18 19:06:11.950538 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.950513    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 19:06:11.955672 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:11.955648    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-351045-7446cc65f7-h8qbx"]
Sep 18 19:06:13.433871 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:13.433834    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbb4b3de-ccef-45ed-a602-e055c3f91811" path="/var/lib/kubelet/pods/dbb4b3de-ccef-45ed-a602-e055c3f91811/volumes"
Sep 18 19:06:21.315009 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:21.314975    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:06:34.315488 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:34.315454    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:06:47.511861 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:06:47.511824    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:07:10.215633 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:07:10.215588    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:08:04.713465 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:04.713425    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:08:18.109786 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:18.109743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:08:30.610992 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.610959    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611370    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f653df1-fbbf-4471-8bb0-e219064fa13b" containerName="payload-pre-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611382    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f653df1-fbbf-4471-8bb0-e219064fa13b" containerName="payload-pre-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611396    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" containerName="payload-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611401    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" containerName="payload-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611409    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611417    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611426    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611433    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611442    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" containerName="payload-pre-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611448    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" containerName="payload-pre-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611453    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb4b3de-ccef-45ed-a602-e055c3f91811" containerName="payload-processing"
Sep 18 19:08:30.611452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611458    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb4b3de-ccef-45ed-a602-e055c3f91811" containerName="payload-processing"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611521    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb4b3de-ccef-45ed-a602-e055c3f91811" containerName="payload-processing"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611530    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611536    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fb1ac8-268d-4254-85e9-6a853986ad5c" containerName="payload-pre-processing"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611541    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0bbd206-5489-4521-a0dd-f871c5a2d6d1" containerName="payload-processing"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611548    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f653df1-fbbf-4471-8bb0-e219064fa13b" containerName="payload-pre-processing"
Sep 18 19:08:30.611816 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.611553    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="853c9fcd-c69a-40d4-aeab-2f7113077e37" containerName="istio-proxy"
Sep 18 19:08:30.614799 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.614783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.617417 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.617393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-5855d4-serving-cert\""
Sep 18 19:08:30.624855 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.624832    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:08:30.762156 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.762122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.762337 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.762194    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndfb\" (UniqueName: \"kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.762337 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.762217    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.762337 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.762264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.863518 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.863427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.863518 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.863468    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.863518 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.863511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndfb\" (UniqueName: \"kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.863805 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.863533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.863805 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:08:30.863664    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-serving-cert: secret "maas-api-e2e-ipp-cleanup-5855d4-serving-cert" not found
Sep 18 19:08:30.863805 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:08:30.863783    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls podName:c53d8d25-b2e7-47cf-b80d-1a7b46f93aab nodeName:}" failed. No retries permitted until 2026-09-18 19:08:31.363757264 +0000 UTC m=+1754.383553885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" (UID: "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab") : secret "maas-api-e2e-ipp-cleanup-5855d4-serving-cert" not found
Sep 18 19:08:30.864258 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.864239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.866170 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.866150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:30.871814 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:30.871793    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndfb\" (UniqueName: \"kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:31.316326 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.316282    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:08:31.319792 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.319769    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.322327 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.322301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5855d4-dockercfg-wpgmv\""
Sep 18 19:08:31.322438 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.322338    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5855d4\""
Sep 18 19:08:31.332295 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.332267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:08:31.369095 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.369064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:31.371614 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.371593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:31.470166 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.470127    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wtc\" (UniqueName: \"kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.470329 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.470192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.526755 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.526713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:31.571401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.571311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wtc\" (UniqueName: \"kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.571401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.571387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.572136 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.572112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.581295 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.581238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wtc\" (UniqueName: \"kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc\") pod \"payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.630373 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.630335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:31.808342 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.808310    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:08:31.809237 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:08:31.809213    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53d8d25_b2e7_47cf_b80d_1a7b46f93aab.slice/crio-196bd8ead2750f205766ffeefdd0e538f3fe9513478baffa64bd7a8d8d16e1bc WatchSource:0}: Error finding container 196bd8ead2750f205766ffeefdd0e538f3fe9513478baffa64bd7a8d8d16e1bc: Status 404 returned error can't find the container with id 196bd8ead2750f205766ffeefdd0e538f3fe9513478baffa64bd7a8d8d16e1bc
Sep 18 19:08:31.810709 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.810688    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:08:31.910216 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:31.910162    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:08:31.912525 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:08:31.912492    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131cd1f9_481c_4219_88f3_7b96768d057f.slice/crio-883a40a7fae2b1b9409b1c15337194f3f36aeab1e3ea98643a873548e8b0dd2d WatchSource:0}: Error finding container 883a40a7fae2b1b9409b1c15337194f3f36aeab1e3ea98643a873548e8b0dd2d: Status 404 returned error can't find the container with id 883a40a7fae2b1b9409b1c15337194f3f36aeab1e3ea98643a873548e8b0dd2d
Sep 18 19:08:32.582226 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.582120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" event={"ID":"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab","Type":"ContainerStarted","Data":"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"}
Sep 18 19:08:32.582226 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.582181    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" event={"ID":"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab","Type":"ContainerStarted","Data":"196bd8ead2750f205766ffeefdd0e538f3fe9513478baffa64bd7a8d8d16e1bc"}
Sep 18 19:08:32.582452 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.582302    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:32.583758 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.583736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" event={"ID":"131cd1f9-481c-4219-88f3-7b96768d057f","Type":"ContainerStarted","Data":"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"}
Sep 18 19:08:32.583910 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.583780    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:32.583910 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.583804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" event={"ID":"131cd1f9-481c-4219-88f3-7b96768d057f","Type":"ContainerStarted","Data":"883a40a7fae2b1b9409b1c15337194f3f36aeab1e3ea98643a873548e8b0dd2d"}
Sep 18 19:08:32.614536 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.614479    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" podStartSLOduration=2.135123027 podStartE2EDuration="2.614464201s" totalImagesPullingTime="479.341174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:32.610004019 +0000 UTC m=+1755.629800644" watchObservedRunningTime="2026-09-18 19:08:32.614464201 +0000 UTC m=+1755.634260827"
Sep 18 19:08:32.636371 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:32.636313    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" podStartSLOduration=1.636293771 podStartE2EDuration="1.636293771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:32.635834838 +0000 UTC m=+1755.655631474" watchObservedRunningTime="2026-09-18 19:08:32.636293771 +0000 UTC m=+1755.656090399"
Sep 18 19:08:38.528474 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:38.528435    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:08:38.594602 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:38.594572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:08:43.590690 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:43.590662    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:08:59.845336 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:59.845289    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:08:59.845791 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:59.845556    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" podUID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" containerName="maas-api" containerID="cri-o://139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b" gracePeriod=30
Sep 18 19:08:59.918302 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:59.918266    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:08:59.918538 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:08:59.918511    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" podUID="131cd1f9-481c-4219-88f3-7b96768d057f" containerName="payload-processing" containerID="cri-o://6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918" gracePeriod=30
Sep 18 19:09:00.166346 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.166316    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:09:00.202546 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.202513    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:09:00.231275 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.231229    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca\") pod \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") "
Sep 18 19:09:00.231470 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.231381    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") pod \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") "
Sep 18 19:09:00.231470 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.231419    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kndfb\" (UniqueName: \"kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb\") pod \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") "
Sep 18 19:09:00.231470 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.231447    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs\") pod \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\" (UID: \"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab\") "
Sep 18 19:09:00.232643 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.232527    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" (UID: "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:00.235715 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.235673    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" (UID: "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:09:00.235981 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.235945    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" (UID: "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:09:00.236190 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.236164    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb" (OuterVolumeSpecName: "kube-api-access-kndfb") pod "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" (UID: "c53d8d25-b2e7-47cf-b80d-1a7b46f93aab"). InnerVolumeSpecName "kube-api-access-kndfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:00.333043 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333010    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9wtc\" (UniqueName: \"kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc\") pod \"131cd1f9-481c-4219-88f3-7b96768d057f\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") "
Sep 18 19:09:00.333247 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333064    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume\") pod \"131cd1f9-481c-4219-88f3-7b96768d057f\" (UID: \"131cd1f9-481c-4219-88f3-7b96768d057f\") "
Sep 18 19:09:00.333334 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333307    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.333389 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333331    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.333389 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333345    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kndfb\" (UniqueName: \"kubernetes.io/projected/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-kube-api-access-kndfb\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.333389 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333358    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.333490 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.333408    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "131cd1f9-481c-4219-88f3-7b96768d057f" (UID: "131cd1f9-481c-4219-88f3-7b96768d057f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:00.335278 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.335258    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc" (OuterVolumeSpecName: "kube-api-access-l9wtc") pod "131cd1f9-481c-4219-88f3-7b96768d057f" (UID: "131cd1f9-481c-4219-88f3-7b96768d057f"). InnerVolumeSpecName "kube-api-access-l9wtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:00.434103 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.434011    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9wtc\" (UniqueName: \"kubernetes.io/projected/131cd1f9-481c-4219-88f3-7b96768d057f-kube-api-access-l9wtc\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.434103 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.434039    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131cd1f9-481c-4219-88f3-7b96768d057f-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.725615 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.725517    2558 generic.go:356] "Generic (PLEG): container finished" podID="131cd1f9-481c-4219-88f3-7b96768d057f" containerID="6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918" exitCode=0
Sep 18 19:09:00.725615 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.725550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" event={"ID":"131cd1f9-481c-4219-88f3-7b96768d057f","Type":"ContainerDied","Data":"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"}
Sep 18 19:09:00.725615 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.725567    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"
Sep 18 19:09:00.725615 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.725610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf" event={"ID":"131cd1f9-481c-4219-88f3-7b96768d057f","Type":"ContainerDied","Data":"883a40a7fae2b1b9409b1c15337194f3f36aeab1e3ea98643a873548e8b0dd2d"}
Sep 18 19:09:00.726000 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.725637    2558 scope.go:117] "RemoveContainer" containerID="6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"
Sep 18 19:09:00.727123 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.727099    2558 generic.go:356] "Generic (PLEG): container finished" podID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" containerID="139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b" exitCode=0
Sep 18 19:09:00.727243 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.727143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" event={"ID":"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab","Type":"ContainerDied","Data":"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"}
Sep 18 19:09:00.727243 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.727153    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"
Sep 18 19:09:00.727243 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.727178    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t" event={"ID":"c53d8d25-b2e7-47cf-b80d-1a7b46f93aab","Type":"ContainerDied","Data":"196bd8ead2750f205766ffeefdd0e538f3fe9513478baffa64bd7a8d8d16e1bc"}
Sep 18 19:09:00.735673 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.735647    2558 scope.go:117] "RemoveContainer" containerID="6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"
Sep 18 19:09:00.736037 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:09:00.736011    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918\": container with ID starting with 6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918 not found: ID does not exist" containerID="6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"
Sep 18 19:09:00.736126 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.736045    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918"} err="failed to get container status \"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918\": rpc error: code = NotFound desc = could not find container \"6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918\": container with ID starting with 6c0c70c9d121524b58ca777de921e5dcfcf65574886018ec5a7c1cb1f006c918 not found: ID does not exist"
Sep 18 19:09:00.736126 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.736071    2558 scope.go:117] "RemoveContainer" containerID="139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"
Sep 18 19:09:00.744181 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.744162    2558 scope.go:117] "RemoveContainer" containerID="139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"
Sep 18 19:09:00.744444 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:09:00.744424    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b\": container with ID starting with 139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b not found: ID does not exist" containerID="139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"
Sep 18 19:09:00.744508 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.744456    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b"} err="failed to get container status \"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b\": rpc error: code = NotFound desc = could not find container \"139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b\": container with ID starting with 139239b2a6253f4cb4b97f0dfb27e352012a8bb619d06919ae56390318a2ef5b not found: ID does not exist"
Sep 18 19:09:00.753108 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.753076    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:09:00.760854 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.760832    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5855d4-578dcf9bb5-4ssgf"]
Sep 18 19:09:00.795094 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.795062    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:09:00.809638 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:00.809602    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5855d4-6ff78bf6d7-rhc8t"]
Sep 18 19:09:01.434074 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:01.434035    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="131cd1f9-481c-4219-88f3-7b96768d057f" path="/var/lib/kubelet/pods/131cd1f9-481c-4219-88f3-7b96768d057f/volumes"
Sep 18 19:09:01.434465 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:01.434447    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" path="/var/lib/kubelet/pods/c53d8d25-b2e7-47cf-b80d-1a7b46f93aab/volumes"
Sep 18 19:09:04.540203 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:04.540170    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:09:24.617144 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:24.617105    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:09:37.720222 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:09:37.720186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:10:30.609845 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:30.609804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:10:45.770035 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:45.769939    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 19:10:45.770814 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:45.770761    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" podUID="44485db6-8f2a-4187-b46f-0540a303de0d" containerName="maas-api" containerID="cri-o://06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6" gracePeriod=30
Sep 18 19:10:45.827826 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:45.827789    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 19:10:45.828080 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:45.828049    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" podUID="3430d71e-bcf2-452c-b912-c5613fd0948a" containerName="payload-processing" containerID="cri-o://6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2" gracePeriod=30
Sep 18 19:10:46.080794 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.080770    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 19:10:46.123451 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.123420    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 19:10:46.191137 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191106    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs\") pod \"44485db6-8f2a-4187-b46f-0540a303de0d\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") "
Sep 18 19:10:46.191137 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191143    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nbc2\" (UniqueName: \"kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2\") pod \"44485db6-8f2a-4187-b46f-0540a303de0d\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") "
Sep 18 19:10:46.191401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191172    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca\") pod \"44485db6-8f2a-4187-b46f-0540a303de0d\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") "
Sep 18 19:10:46.191401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191197    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") pod \"44485db6-8f2a-4187-b46f-0540a303de0d\" (UID: \"44485db6-8f2a-4187-b46f-0540a303de0d\") "
Sep 18 19:10:46.191401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume\") pod \"3430d71e-bcf2-452c-b912-c5613fd0948a\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") "
Sep 18 19:10:46.191401 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191387    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk7gb\" (UniqueName: \"kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb\") pod \"3430d71e-bcf2-452c-b912-c5613fd0948a\" (UID: \"3430d71e-bcf2-452c-b912-c5613fd0948a\") "
Sep 18 19:10:46.191618 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191581    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44485db6-8f2a-4187-b46f-0540a303de0d" (UID: "44485db6-8f2a-4187-b46f-0540a303de0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:46.191744 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.191718    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44485db6-8f2a-4187-b46f-0540a303de0d-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.192240 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.192185    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3430d71e-bcf2-452c-b912-c5613fd0948a" (UID: "3430d71e-bcf2-452c-b912-c5613fd0948a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:46.193595 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.193573    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44485db6-8f2a-4187-b46f-0540a303de0d" (UID: "44485db6-8f2a-4187-b46f-0540a303de0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:10:46.193595 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.193576    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2" (OuterVolumeSpecName: "kube-api-access-6nbc2") pod "44485db6-8f2a-4187-b46f-0540a303de0d" (UID: "44485db6-8f2a-4187-b46f-0540a303de0d"). InnerVolumeSpecName "kube-api-access-6nbc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:46.194046 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.194026    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "44485db6-8f2a-4187-b46f-0540a303de0d" (UID: "44485db6-8f2a-4187-b46f-0540a303de0d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:10:46.194126 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.194046    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb" (OuterVolumeSpecName: "kube-api-access-hk7gb") pod "3430d71e-bcf2-452c-b912-c5613fd0948a" (UID: "3430d71e-bcf2-452c-b912-c5613fd0948a"). InnerVolumeSpecName "kube-api-access-hk7gb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:46.229064 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.229030    2558 generic.go:356] "Generic (PLEG): container finished" podID="44485db6-8f2a-4187-b46f-0540a303de0d" containerID="06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6" exitCode=0
Sep 18 19:10:46.229254 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.229078    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"
Sep 18 19:10:46.229254 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.229105    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" event={"ID":"44485db6-8f2a-4187-b46f-0540a303de0d","Type":"ContainerDied","Data":"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"}
Sep 18 19:10:46.229254 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.229152    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw" event={"ID":"44485db6-8f2a-4187-b46f-0540a303de0d","Type":"ContainerDied","Data":"4e69ce5adbb4d63e0859ae61f009f25f2d2e7881688a67227733dac23cdc733d"}
Sep 18 19:10:46.229254 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.229172    2558 scope.go:117] "RemoveContainer" containerID="06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"
Sep 18 19:10:46.230508 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.230477    2558 generic.go:356] "Generic (PLEG): container finished" podID="3430d71e-bcf2-452c-b912-c5613fd0948a" containerID="6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2" exitCode=0
Sep 18 19:10:46.230618 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.230522    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"
Sep 18 19:10:46.230618 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.230526    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" event={"ID":"3430d71e-bcf2-452c-b912-c5613fd0948a","Type":"ContainerDied","Data":"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"}
Sep 18 19:10:46.230829 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.230629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm" event={"ID":"3430d71e-bcf2-452c-b912-c5613fd0948a","Type":"ContainerDied","Data":"0bf6bbc36a23c9b35031236cc2a4fb9d08ba8c60ace05147eafff5a17446917c"}
Sep 18 19:10:46.238015 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.237988    2558 scope.go:117] "RemoveContainer" containerID="06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"
Sep 18 19:10:46.238362 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:10:46.238290    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6\": container with ID starting with 06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6 not found: ID does not exist" containerID="06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"
Sep 18 19:10:46.238362 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.238323    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6"} err="failed to get container status \"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6\": rpc error: code = NotFound desc = could not find container \"06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6\": container with ID starting with 06aa488a1d151520e7fae98d98833cb6269795556ce63c6c35aefaa9281b23e6 not found: ID does not exist"
Sep 18 19:10:46.238362 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.238362    2558 scope.go:117] "RemoveContainer" containerID="6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"
Sep 18 19:10:46.248825 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.248271    2558 scope.go:117] "RemoveContainer" containerID="6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"
Sep 18 19:10:46.248825 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:10:46.248743    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2\": container with ID starting with 6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2 not found: ID does not exist" containerID="6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"
Sep 18 19:10:46.248825 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.248776    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2"} err="failed to get container status \"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2\": rpc error: code = NotFound desc = could not find container \"6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2\": container with ID starting with 6e3c5b1b17603648b0d7017793ce6f5ab8f1e71ba4f141cebe5251c90e336ec2 not found: ID does not exist"
Sep 18 19:10:46.253550 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.253525    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 19:10:46.255647 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.255626    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-298125-799bf4468c-k58dm"]
Sep 18 19:10:46.266747 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.266725    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 19:10:46.269196 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.269174    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-298125-7769f78bf8-8d4zw"]
Sep 18 19:10:46.293142 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.293115    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.293142 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.293143    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nbc2\" (UniqueName: \"kubernetes.io/projected/44485db6-8f2a-4187-b46f-0540a303de0d-kube-api-access-6nbc2\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.293295 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.293153    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44485db6-8f2a-4187-b46f-0540a303de0d-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.293295 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.293169    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3430d71e-bcf2-452c-b912-c5613fd0948a-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.293295 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:46.293180    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk7gb\" (UniqueName: \"kubernetes.io/projected/3430d71e-bcf2-452c-b912-c5613fd0948a-kube-api-access-hk7gb\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:47.433941 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:47.433906    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3430d71e-bcf2-452c-b912-c5613fd0948a" path="/var/lib/kubelet/pods/3430d71e-bcf2-452c-b912-c5613fd0948a/volumes"
Sep 18 19:10:47.434328 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:47.434317    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44485db6-8f2a-4187-b46f-0540a303de0d" path="/var/lib/kubelet/pods/44485db6-8f2a-4187-b46f-0540a303de0d/volumes"
Sep 18 19:10:48.805063 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:48.805030    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:10:52.626105 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:52.626048    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 19:10:52.626554 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:52.626314    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" containerID="cri-o://50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a" gracePeriod=30
Sep 18 19:10:53.299847 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:53.299802    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:10:53.300063 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:53.299871    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:10:57.888688 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:57.888658    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 19:10:58.000026 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:57.999923    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000026 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:57.999987    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000026 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000015    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000041    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000060    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000101    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000185    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rw6f\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000218    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token\") pod \"92ee097d-9630-400b-a66f-0dde1e81333e\" (UID: \"92ee097d-9630-400b-a66f-0dde1e81333e\") "
Sep 18 19:10:58.000315 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000239    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000357    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000469    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data" (OuterVolumeSpecName: "istio-data") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000502    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000509    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000538    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-credential-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.000670 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.000549    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-workload-socket\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.002529 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.002501    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f" (OuterVolumeSpecName: "kube-api-access-5rw6f") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "kube-api-access-5rw6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:58.002529 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.002503    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:10:58.002996 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.002964    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:58.002996 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.002979    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token" (OuterVolumeSpecName: "istio-token") pod "92ee097d-9630-400b-a66f-0dde1e81333e" (UID: "92ee097d-9630-400b-a66f-0dde1e81333e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:58.101946 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101873    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-istio-token\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.101946 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101939    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ee097d-9630-400b-a66f-0dde1e81333e-istio-podinfo\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.101946 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101951    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ee097d-9630-400b-a66f-0dde1e81333e-istiod-ca-cert\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.102209 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101963    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-envoy\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.102209 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101974    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ee097d-9630-400b-a66f-0dde1e81333e-istio-data\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.102209 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.101984    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rw6f\" (UniqueName: \"kubernetes.io/projected/92ee097d-9630-400b-a66f-0dde1e81333e-kube-api-access-5rw6f\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:10:58.290376 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.290339    2558 generic.go:356] "Generic (PLEG): container finished" podID="92ee097d-9630-400b-a66f-0dde1e81333e" containerID="50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a" exitCode=0
Sep 18 19:10:58.290547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.290389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"
Sep 18 19:10:58.290547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.290396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerDied","Data":"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"}
Sep 18 19:10:58.290547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.290463    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4" event={"ID":"92ee097d-9630-400b-a66f-0dde1e81333e","Type":"ContainerDied","Data":"56601ef6220e306977af5280f8e438c16057986dff781743bf92012f66cb1bc1"}
Sep 18 19:10:58.290547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.290484    2558 scope.go:117] "RemoveContainer" containerID="50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"
Sep 18 19:10:58.299374 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.299357    2558 scope.go:117] "RemoveContainer" containerID="59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"
Sep 18 19:10:58.307507 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.307487    2558 scope.go:117] "RemoveContainer" containerID="50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"
Sep 18 19:10:58.307813 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:10:58.307793    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a\": container with ID starting with 50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a not found: ID does not exist" containerID="50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"
Sep 18 19:10:58.307858 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.307823    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a"} err="failed to get container status \"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a\": rpc error: code = NotFound desc = could not find container \"50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a\": container with ID starting with 50f87a6392adc4192de5021f1661354a5b3ea1dc81e82daf731c76dedcc5a25a not found: ID does not exist"
Sep 18 19:10:58.307858 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.307846    2558 scope.go:117] "RemoveContainer" containerID="59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"
Sep 18 19:10:58.308117 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:10:58.308102    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2\": container with ID starting with 59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2 not found: ID does not exist" containerID="59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"
Sep 18 19:10:58.308158 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.308121    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2"} err="failed to get container status \"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2\": rpc error: code = NotFound desc = could not find container \"59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2\": container with ID starting with 59654772dbfabfb9204a8cd4ee83fd910548699775176174e0c6b92c8d6912b2 not found: ID does not exist"
Sep 18 19:10:58.312431 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.312406    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 19:10:58.316684 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:58.316659    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-298125-openshift-default-697b9bf4dd-85js4"]
Sep 18 19:10:59.433806 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:10:59.433774    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" path="/var/lib/kubelet/pods/92ee097d-9630-400b-a66f-0dde1e81333e/volumes"
Sep 18 19:11:03.489238 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.489204    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 19:11:03.489860 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.489437    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" containerName="maas-api" containerID="cri-o://7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4" gracePeriod=30
Sep 18 19:11:03.524571 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.524537    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"]
Sep 18 19:11:03.526737 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.526711    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-a6d509-29829300-lgm55"]
Sep 18 19:11:03.552572 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.552537    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 19:11:03.552824 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.552799    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" podUID="78a52c10-ee96-48ff-b81a-469b63028c5e" containerName="payload-processing" containerID="cri-o://36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995" gracePeriod=30
Sep 18 19:11:03.804567 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.804539    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 19:11:03.838789 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.838761    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 19:11:03.853904 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.853863    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs\") pod \"f34049c5-7a60-453c-a9f6-536c7754d315\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") "
Sep 18 19:11:03.854066 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.853982    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t787p\" (UniqueName: \"kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p\") pod \"f34049c5-7a60-453c-a9f6-536c7754d315\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") "
Sep 18 19:11:03.854066 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854003    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume\") pod \"78a52c10-ee96-48ff-b81a-469b63028c5e\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") "
Sep 18 19:11:03.854066 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854019    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls\") pod \"f34049c5-7a60-453c-a9f6-536c7754d315\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") "
Sep 18 19:11:03.854066 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854045    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttxgg\" (UniqueName: \"kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg\") pod \"78a52c10-ee96-48ff-b81a-469b63028c5e\" (UID: \"78a52c10-ee96-48ff-b81a-469b63028c5e\") "
Sep 18 19:11:03.854289 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854091    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca\") pod \"f34049c5-7a60-453c-a9f6-536c7754d315\" (UID: \"f34049c5-7a60-453c-a9f6-536c7754d315\") "
Sep 18 19:11:03.854562 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78a52c10-ee96-48ff-b81a-469b63028c5e" (UID: "78a52c10-ee96-48ff-b81a-469b63028c5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:11:03.854665 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.854609    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f34049c5-7a60-453c-a9f6-536c7754d315" (UID: "f34049c5-7a60-453c-a9f6-536c7754d315"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:11:03.856381 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.856352    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f34049c5-7a60-453c-a9f6-536c7754d315" (UID: "f34049c5-7a60-453c-a9f6-536c7754d315"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:03.856468 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.856422    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p" (OuterVolumeSpecName: "kube-api-access-t787p") pod "f34049c5-7a60-453c-a9f6-536c7754d315" (UID: "f34049c5-7a60-453c-a9f6-536c7754d315"). InnerVolumeSpecName "kube-api-access-t787p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:03.856713 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.856688    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f34049c5-7a60-453c-a9f6-536c7754d315" (UID: "f34049c5-7a60-453c-a9f6-536c7754d315"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:03.856713 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.856697    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg" (OuterVolumeSpecName: "kube-api-access-ttxgg") pod "78a52c10-ee96-48ff-b81a-469b63028c5e" (UID: "78a52c10-ee96-48ff-b81a-469b63028c5e"). InnerVolumeSpecName "kube-api-access-ttxgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:03.954759 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954725    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t787p\" (UniqueName: \"kubernetes.io/projected/f34049c5-7a60-453c-a9f6-536c7754d315-kube-api-access-t787p\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:03.954759 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954755    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a52c10-ee96-48ff-b81a-469b63028c5e-plugins-config-volume\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:03.954759 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954765    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:03.955029 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954775    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttxgg\" (UniqueName: \"kubernetes.io/projected/78a52c10-ee96-48ff-b81a-469b63028c5e-kube-api-access-ttxgg\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:03.955029 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954784    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34049c5-7a60-453c-a9f6-536c7754d315-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:03.955029 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:03.954792    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34049c5-7a60-453c-a9f6-536c7754d315-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:11:04.318555 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.318519    2558 generic.go:356] "Generic (PLEG): container finished" podID="f34049c5-7a60-453c-a9f6-536c7754d315" containerID="7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4" exitCode=0
Sep 18 19:11:04.318728 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.318554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" event={"ID":"f34049c5-7a60-453c-a9f6-536c7754d315","Type":"ContainerDied","Data":"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"}
Sep 18 19:11:04.318728 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.318611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" event={"ID":"f34049c5-7a60-453c-a9f6-536c7754d315","Type":"ContainerDied","Data":"8d8295a9984c0a85e4b5f4fcc57a7a3dcb00b0ef6a3bb63158b79dfaefc324e6"}
Sep 18 19:11:04.318728 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.318634    2558 scope.go:117] "RemoveContainer" containerID="7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"
Sep 18 19:11:04.318728 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.318570    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"
Sep 18 19:11:04.319995 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.319972    2558 generic.go:356] "Generic (PLEG): container finished" podID="78a52c10-ee96-48ff-b81a-469b63028c5e" containerID="36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995" exitCode=0
Sep 18 19:11:04.320082 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.320021    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" event={"ID":"78a52c10-ee96-48ff-b81a-469b63028c5e","Type":"ContainerDied","Data":"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"}
Sep 18 19:11:04.320082 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.320060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb" event={"ID":"78a52c10-ee96-48ff-b81a-469b63028c5e","Type":"ContainerDied","Data":"0fc68864cbda2f5f65b78cdd44056d32370c9282921f438ebacd3589ca4b3ccb"}
Sep 18 19:11:04.320082 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.320024    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"
Sep 18 19:11:04.329558 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.329341    2558 scope.go:117] "RemoveContainer" containerID="7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"
Sep 18 19:11:04.329884 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:11:04.329858    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4\": container with ID starting with 7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4 not found: ID does not exist" containerID="7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"
Sep 18 19:11:04.330072 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.330033    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4"} err="failed to get container status \"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4\": rpc error: code = NotFound desc = could not find container \"7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4\": container with ID starting with 7a664a4c61c5231bd6dac498bd74103e7fe60d0752ce0c2210787122c39b7cc4 not found: ID does not exist"
Sep 18 19:11:04.330072 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.330073    2558 scope.go:117] "RemoveContainer" containerID="36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"
Sep 18 19:11:04.338135 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.338117    2558 scope.go:117] "RemoveContainer" containerID="36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"
Sep 18 19:11:04.338426 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:11:04.338407    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995\": container with ID starting with 36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995 not found: ID does not exist" containerID="36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"
Sep 18 19:11:04.338480 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.338434    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995"} err="failed to get container status \"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995\": rpc error: code = NotFound desc = could not find container \"36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995\": container with ID starting with 36d85406d7a01cd05d9f4f9e0ec5d6d0426c4e37cb6edbd69d147c3cf4342995 not found: ID does not exist"
Sep 18 19:11:04.347540 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.347509    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 19:11:04.353696 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.353671    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh"]
Sep 18 19:11:04.364627 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.364601    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 19:11:04.368637 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:04.368614    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a6d509-54694dd5f7-jdplb"]
Sep 18 19:11:05.433720 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:05.433685    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78a52c10-ee96-48ff-b81a-469b63028c5e" path="/var/lib/kubelet/pods/78a52c10-ee96-48ff-b81a-469b63028c5e/volumes"
Sep 18 19:11:05.434128 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:05.434092    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" path="/var/lib/kubelet/pods/f34049c5-7a60-453c-a9f6-536c7754d315/volumes"
Sep 18 19:11:05.434339 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:05.434326    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d95074-cc6d-490d-8174-dd39c3823472" path="/var/lib/kubelet/pods/f9d95074-cc6d-490d-8174-dd39c3823472/volumes"
Sep 18 19:11:06.304640 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:06.304607    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:11:06.761161 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:06.761053    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a6d509-fb959cb97-d2slh" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.50:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 18 19:11:14.701486 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:14.701442    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:11:17.628034 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:11:17.627998    2558 scope.go:117] "RemoveContainer" containerID="c42c2b547b76fabba3c8bb969b11f7c0282aafc2c2101df892532346da84ff83"
Sep 18 19:12:35.901907 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:12:35.901865    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:13:02.205132 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:13:02.205096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:13:09.916454 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:13:09.916420    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:13:18.198959 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:13:18.198924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:13:35.808535 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:13:35.808503    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:00.145118 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145077    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"]
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145411    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a52c10-ee96-48ff-b81a-469b63028c5e" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145427    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a52c10-ee96-48ff-b81a-469b63028c5e" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145440    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145449    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145456    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3430d71e-bcf2-452c-b912-c5613fd0948a" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145461    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3430d71e-bcf2-452c-b912-c5613fd0948a" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145470    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145476    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145482    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145486    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145493    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44485db6-8f2a-4187-b46f-0540a303de0d" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145498    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="44485db6-8f2a-4187-b46f-0540a303de0d" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145507    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="131cd1f9-481c-4219-88f3-7b96768d057f" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145512    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="131cd1f9-481c-4219-88f3-7b96768d057f" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145563    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145570    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c53d8d25-b2e7-47cf-b80d-1a7b46f93aab" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145577    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f34049c5-7a60-453c-a9f6-536c7754d315" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145583    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145589    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="44485db6-8f2a-4187-b46f-0540a303de0d" containerName="maas-api"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145596    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="78a52c10-ee96-48ff-b81a-469b63028c5e" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145601    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="131cd1f9-481c-4219-88f3-7b96768d057f" containerName="payload-processing"
Sep 18 19:15:00.147765 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.145606    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3430d71e-bcf2-452c-b912-c5613fd0948a" containerName="payload-processing"
Sep 18 19:15:00.148550 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.148370    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"
Sep 18 19:15:00.150988 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.150966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xvbnc\""
Sep 18 19:15:00.157356 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.157316    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"]
Sep 18 19:15:00.159041 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.159022    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"
Sep 18 19:15:00.434632 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:00.434547    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"]
Sep 18 19:15:00.438263 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:15:00.438235    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7feef2a_2449_427e_956b_a0aa0626dab6.slice/crio-84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc WatchSource:0}: Error finding container 84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc: Status 404 returned error can't find the container with id 84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc
Sep 18 19:15:01.338128 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:01.338098    2558 generic.go:356] "Generic (PLEG): container finished" podID="f7feef2a-2449-427e-956b-a0aa0626dab6" containerID="3845b7575c86db1f0e21d7d542bae92f87e8e3eccc230ae2237a0caada60a934" exitCode=0
Sep 18 19:15:01.338510 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:01.338143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d" event={"ID":"f7feef2a-2449-427e-956b-a0aa0626dab6","Type":"ContainerDied","Data":"3845b7575c86db1f0e21d7d542bae92f87e8e3eccc230ae2237a0caada60a934"}
Sep 18 19:15:01.338510 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:01.338181    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d" event={"ID":"f7feef2a-2449-427e-956b-a0aa0626dab6","Type":"ContainerStarted","Data":"84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc"}
Sep 18 19:15:02.472972 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:02.472938    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"
Sep 18 19:15:02.606475 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:02.606377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:03.349125 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:03.349093    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d"
Sep 18 19:15:03.349125 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:03.349120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-78g2d" event={"ID":"f7feef2a-2449-427e-956b-a0aa0626dab6","Type":"ContainerDied","Data":"84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc"}
Sep 18 19:15:03.349386 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:03.349168    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84a9257666b50c243047f77d4cb68083e9e15b088660fa650e4a3d4ecba338bc"
Sep 18 19:15:09.417434 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:09.417346    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:18.812239 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:18.812198    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:23.311814 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:23.311778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:34.308484 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:34.308444    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:15:57.035926 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:15:57.034879    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:03.003755 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:03.003713    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:10.017564 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.017516    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:10.258070 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258025    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:10.258392 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258380    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:16:10.258442 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258396    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ee097d-9630-400b-a66f-0dde1e81333e" containerName="istio-proxy"
Sep 18 19:16:10.258442 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258410    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7feef2a-2449-427e-956b-a0aa0626dab6" containerName="cleanup"
Sep 18 19:16:10.258442 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258415    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7feef2a-2449-427e-956b-a0aa0626dab6" containerName="cleanup"
Sep 18 19:16:10.258536 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.258475    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7feef2a-2449-427e-956b-a0aa0626dab6" containerName="cleanup"
Sep 18 19:16:10.261615 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.261588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.264360 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.264330    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 19:16:10.265226 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.265205    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 19:16:10.270437 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.270416    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:16:10.270744 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.270690    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-htbnh\""
Sep 18 19:16:10.270871 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.270851    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 19:16:10.291160 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.291126    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:10.417451 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.417401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.417653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.417470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r88w2\" (UniqueName: \"kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.417653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.417521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.417653 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.417549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.518722 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.518667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.518722 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.518728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.519051 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.518789    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r88w2\" (UniqueName: \"kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.519051 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.518840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.519442 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.519375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.521432 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.521412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.521526 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.521461    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.527180 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.527154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r88w2\" (UniqueName: \"kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2\") pod \"maas-api-59f6566bbc-7drrs\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.574216 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.574175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:10.854105 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.853934    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:10.856310 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:16:10.856269    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa336599_8ce7_4b48_bf5b_f7d2f6ba93ce.slice/crio-d8398bd8b349cf105ff87721f2997c264b499bf0fe9e2a7d45f51af47361e3f6 WatchSource:0}: Error finding container d8398bd8b349cf105ff87721f2997c264b499bf0fe9e2a7d45f51af47361e3f6: Status 404 returned error can't find the container with id d8398bd8b349cf105ff87721f2997c264b499bf0fe9e2a7d45f51af47361e3f6
Sep 18 19:16:10.860668 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:10.860643    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:16:11.655694 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:11.655648    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" event={"ID":"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce","Type":"ContainerStarted","Data":"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"}
Sep 18 19:16:11.656209 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:11.655708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" event={"ID":"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce","Type":"ContainerStarted","Data":"d8398bd8b349cf105ff87721f2997c264b499bf0fe9e2a7d45f51af47361e3f6"}
Sep 18 19:16:11.656209 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:11.655730    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:11.677547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:11.677491    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" podStartSLOduration=1.361584426 podStartE2EDuration="1.677471482s" totalImagesPullingTime="315.887056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:11.675330028 +0000 UTC m=+2214.695126666" watchObservedRunningTime="2026-09-18 19:16:11.677471482 +0000 UTC m=+2214.697268108"
Sep 18 19:16:17.347844 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.347804    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:17.348338 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.348120    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" podUID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" containerName="maas-api" containerID="cri-o://4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260" gracePeriod=30
Sep 18 19:16:17.354429 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.354403    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:17.382436 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.382401    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"]
Sep 18 19:16:17.385516 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.385495    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.395719 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.395678    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"]
Sep 18 19:16:17.483396 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.483346    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-maas-api-tls\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.483592 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.483451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-metrics-certs\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.483592 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.483495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e275e5-5418-4f7d-8cd5-c3074787f7c0-openshift-service-ca\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.483592 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.483519    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-424k2\" (UniqueName: \"kubernetes.io/projected/06e275e5-5418-4f7d-8cd5-c3074787f7c0-kube-api-access-424k2\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.584630 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.584596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-metrics-certs\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.584775 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.584664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e275e5-5418-4f7d-8cd5-c3074787f7c0-openshift-service-ca\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.584775 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.584696    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-424k2\" (UniqueName: \"kubernetes.io/projected/06e275e5-5418-4f7d-8cd5-c3074787f7c0-kube-api-access-424k2\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.584775 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.584767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-maas-api-tls\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.585878 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.585851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e275e5-5418-4f7d-8cd5-c3074787f7c0-openshift-service-ca\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.587463 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.587440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-maas-api-tls\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.587567 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.587508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06e275e5-5418-4f7d-8cd5-c3074787f7c0-metrics-certs\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.594305 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.594279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-424k2\" (UniqueName: \"kubernetes.io/projected/06e275e5-5418-4f7d-8cd5-c3074787f7c0-kube-api-access-424k2\") pod \"maas-api-86d9ddc875-lk9b6\" (UID: \"06e275e5-5418-4f7d-8cd5-c3074787f7c0\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.612752 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.612675    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:17.684958 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.684916    2558 generic.go:356] "Generic (PLEG): container finished" podID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" containerID="4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260" exitCode=0
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.684977    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" event={"ID":"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce","Type":"ContainerDied","Data":"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"}
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685080    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs" event={"ID":"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce","Type":"ContainerDied","Data":"d8398bd8b349cf105ff87721f2997c264b499bf0fe9e2a7d45f51af47361e3f6"}
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685101    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca\") pod \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") "
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685110    2558 scope.go:117] "RemoveContainer" containerID="4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685149    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls\") pod \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") "
Sep 18 19:16:17.685173 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685171    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs\") pod \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") "
Sep 18 19:16:17.685556 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r88w2\" (UniqueName: \"kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2\") pod \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\" (UID: \"fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce\") "
Sep 18 19:16:17.685556 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.685546    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" (UID: "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:16:17.688086 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.687998    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2" (OuterVolumeSpecName: "kube-api-access-r88w2") pod "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" (UID: "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce"). InnerVolumeSpecName "kube-api-access-r88w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:16:17.688201 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.688107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" (UID: "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:17.688201 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.688116    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" (UID: "fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:17.694420 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.694404    2558 scope.go:117] "RemoveContainer" containerID="4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"
Sep 18 19:16:17.694717 ip-10-0-132-229 kubenswrapper[2558]: E0918 19:16:17.694697    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260\": container with ID starting with 4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260 not found: ID does not exist" containerID="4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"
Sep 18 19:16:17.694784 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.694725    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260"} err="failed to get container status \"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260\": rpc error: code = NotFound desc = could not find container \"4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260\": container with ID starting with 4cdb9c2050f7009b766b5b05a785c7272c777a910ab1e8d3da8002af0d992260 not found: ID does not exist"
Sep 18 19:16:17.731020 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.730981    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:17.787756 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.787663    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-openshift-service-ca\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:16:17.787756 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.787701    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-maas-api-tls\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:16:17.787756 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.787717    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-metrics-certs\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:16:17.787756 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.787731    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r88w2\" (UniqueName: \"kubernetes.io/projected/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce-kube-api-access-r88w2\") on node \"ip-10-0-132-229.ec2.internal\" DevicePath \"\""
Sep 18 19:16:17.911092 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:17.910998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:18.004307 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.004249    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"]
Sep 18 19:16:18.006602 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:16:18.006565    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e275e5_5418_4f7d_8cd5_c3074787f7c0.slice/crio-b0bea007dd3e8e51b83ba0b3624a7e654833dfcab8334d0b16ba63a26d418f57 WatchSource:0}: Error finding container b0bea007dd3e8e51b83ba0b3624a7e654833dfcab8334d0b16ba63a26d418f57: Status 404 returned error can't find the container with id b0bea007dd3e8e51b83ba0b3624a7e654833dfcab8334d0b16ba63a26d418f57
Sep 18 19:16:18.008241 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.008220    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:18.012185 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.012164    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7drrs"]
Sep 18 19:16:18.693072 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.692957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6" event={"ID":"06e275e5-5418-4f7d-8cd5-c3074787f7c0","Type":"ContainerStarted","Data":"46663b7b8eb14f22ebf485419a14ec324ef58924ecfae61d9910f2990268458f"}
Sep 18 19:16:18.693072 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.693030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6" event={"ID":"06e275e5-5418-4f7d-8cd5-c3074787f7c0","Type":"ContainerStarted","Data":"b0bea007dd3e8e51b83ba0b3624a7e654833dfcab8334d0b16ba63a26d418f57"}
Sep 18 19:16:18.693072 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.693065    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:18.711616 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:18.711544    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6" podStartSLOduration=1.425565119 podStartE2EDuration="1.711527776s" totalImagesPullingTime="285.962657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:18.710496416 +0000 UTC m=+2221.730293067" watchObservedRunningTime="2026-09-18 19:16:18.711527776 +0000 UTC m=+2221.731324402"
Sep 18 19:16:19.433921 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:19.433872    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" path="/var/lib/kubelet/pods/fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce/volumes"
Sep 18 19:16:24.704078 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:24.703988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-lk9b6"
Sep 18 19:16:24.706591 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:24.706563    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:32.602939 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:32.602903    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:37.807547 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:37.807503    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-fb7v5"]
Sep 18 19:16:44.675289 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:44.675209    2558 ???:1] "http: TLS handshake error from 10.0.130.118:47326: EOF"
Sep 18 19:16:44.683583 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:44.683555    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5htxl_244d630b-d03b-4e98-8f83-81a041d5eaee/global-pull-secret-syncer/0.log"
Sep 18 19:16:44.836838 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:44.836802    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ksjrt_a34cbd08-703c-4101-95de-a5d18ba8a293/konnectivity-agent/0.log"
Sep 18 19:16:44.947697 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:44.947614    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-229.ec2.internal_b315991c662748cf7402dbb21807e910/haproxy/0.log"
Sep 18 19:16:49.163929 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:49.163874    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-d9vqc_a7893c22-dbb2-4423-a25c-6e1d8ed72eff/manager/0.log"
Sep 18 19:16:49.297377 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:49.297342    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-fb7v5_b878abc6-1b2d-47dd-a267-984ff431b006/limitador/0.log"
Sep 18 19:16:51.941143 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:51.941047    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh2rr_0f81b2ec-5918-4a0a-8eeb-440c5dff2d34/node-exporter/0.log"
Sep 18 19:16:51.972579 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:51.972550    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh2rr_0f81b2ec-5918-4a0a-8eeb-440c5dff2d34/kube-rbac-proxy/0.log"
Sep 18 19:16:52.008235 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:52.008207    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh2rr_0f81b2ec-5918-4a0a-8eeb-440c5dff2d34/init-textfile/0.log"
Sep 18 19:16:54.477815 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.477775    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"]
Sep 18 19:16:54.478223 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.478159    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" containerName="maas-api"
Sep 18 19:16:54.478223 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.478171    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" containerName="maas-api"
Sep 18 19:16:54.478223 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.478219    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa336599-8ce7-4b48-bf5b-f7d2f6ba93ce" containerName="maas-api"
Sep 18 19:16:54.481320 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.481298    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.484531 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.484514    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g6kgh\"/\"kube-root-ca.crt\""
Sep 18 19:16:54.484830 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.484806    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g6kgh\"/\"openshift-service-ca.crt\""
Sep 18 19:16:54.485426 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.485398    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g6kgh\"/\"default-dockercfg-xkzgg\""
Sep 18 19:16:54.495970 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.495942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"]
Sep 18 19:16:54.618210 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.618170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-sys\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.618210 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.618212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2bzw\" (UniqueName: \"kubernetes.io/projected/8f5dca75-689f-41fe-a810-748c6a2b6031-kube-api-access-r2bzw\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.618428 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.618258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-proc\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.618428 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.618303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-lib-modules\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.618428 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.618354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-podres\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.719796 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-sys\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.719796 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bzw\" (UniqueName: \"kubernetes.io/projected/8f5dca75-689f-41fe-a810-748c6a2b6031-kube-api-access-r2bzw\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-proc\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719854    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-lib-modules\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719918    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-sys\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-podres\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.719937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-proc\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.720037    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-podres\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.720067 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.720058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f5dca75-689f-41fe-a810-748c6a2b6031-lib-modules\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.730635 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.730563    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2bzw\" (UniqueName: \"kubernetes.io/projected/8f5dca75-689f-41fe-a810-748c6a2b6031-kube-api-access-r2bzw\") pod \"perf-node-gather-daemonset-9vvzw\" (UID: \"8f5dca75-689f-41fe-a810-748c6a2b6031\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:54.790849 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:54.790817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:55.074575 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:55.074546    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"]
Sep 18 19:16:55.077325 ip-10-0-132-229 kubenswrapper[2558]: W0918 19:16:55.077289    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8f5dca75_689f_41fe_a810_748c6a2b6031.slice/crio-3d68736796ad4113422ff99248f98be968025e080fc377c3fb4b7ed766a06ea0 WatchSource:0}: Error finding container 3d68736796ad4113422ff99248f98be968025e080fc377c3fb4b7ed766a06ea0: Status 404 returned error can't find the container with id 3d68736796ad4113422ff99248f98be968025e080fc377c3fb4b7ed766a06ea0
Sep 18 19:16:55.867154 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:55.867115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw" event={"ID":"8f5dca75-689f-41fe-a810-748c6a2b6031","Type":"ContainerStarted","Data":"e03e5e24d97637ddba5d3dc9064995ab54ef03be35d3dbe25ee42910019e463b"}
Sep 18 19:16:55.867555 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:55.867180    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw" event={"ID":"8f5dca75-689f-41fe-a810-748c6a2b6031","Type":"ContainerStarted","Data":"3d68736796ad4113422ff99248f98be968025e080fc377c3fb4b7ed766a06ea0"}
Sep 18 19:16:55.867555 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:55.867241    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:16:55.887325 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:55.887270    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw" podStartSLOduration=1.8872527780000001 podStartE2EDuration="1.887252778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:55.88501708 +0000 UTC m=+2258.904813706" watchObservedRunningTime="2026-09-18 19:16:55.887252778 +0000 UTC m=+2258.907049404"
Sep 18 19:16:57.872786 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:57.872748    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nxqvl_5d636b83-5cbf-40f3-9abf-c312f1fbb6ac/dns/0.log"
Sep 18 19:16:57.916382 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:57.916348    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nxqvl_5d636b83-5cbf-40f3-9abf-c312f1fbb6ac/kube-rbac-proxy/0.log"
Sep 18 19:16:58.108374 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:58.108344    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mcwsf_2d96ac10-d402-43a3-9aea-2360b5f9f206/dns-node-resolver/0.log"
Sep 18 19:16:58.846023 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:58.845987    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-687dbbd5d8-hvg4r_872f73e0-7d28-4967-8473-6276d1c7ebbd/registry/0.log"
Sep 18 19:16:58.879146 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:16:58.879110    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hphd9_f4e9ae49-7084-4a19-a3d2-9c5fc731343b/node-ca/0.log"
Sep 18 19:17:00.749383 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:00.749343    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tzknv_de0c6b90-a1ab-467f-ae6d-c9b435756071/discovery/0.log"
Sep 18 19:17:00.789443 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:00.789409    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-6lzls_cf2c4836-df42-44a7-a965-3ec85386f45e/kube-auth-proxy/0.log"
Sep 18 19:17:00.881053 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:00.881022    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/0.log"
Sep 18 19:17:00.885400 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:00.885367    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/1.log"
Sep 18 19:17:01.884807 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:01.884779    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-9vvzw"
Sep 18 19:17:01.968104 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:01.968059    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dxrcs_fb7d786a-44b9-4939-99eb-07c8655f9b08/serve-healthcheck-canary/0.log"
Sep 18 19:17:02.553863 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:02.553829    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8gwhh_48432861-84d0-48ef-a69c-f3e49ca4110d/kube-rbac-proxy/0.log"
Sep 18 19:17:02.580438 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:02.580409    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8gwhh_48432861-84d0-48ef-a69c-f3e49ca4110d/exporter/0.log"
Sep 18 19:17:02.603092 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:02.603061    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8gwhh_48432861-84d0-48ef-a69c-f3e49ca4110d/extractor/0.log"
Sep 18 19:17:05.349000 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:05.348965    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-54446658c9-8fzsh_9de584c2-f80c-43a5-9e84-16bef23811fc/manager/0.log"
Sep 18 19:17:05.816749 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:05.816710    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4687t_2e428cc9-49f8-49c4-a23d-a9f769b5bb58/manager/0.log"
Sep 18 19:17:05.926440 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:05.926410    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8jhwg_c46fce34-bd2a-4c7b-9e21-e837fc3595c9/server/0.log"
Sep 18 19:17:05.964966 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:05.964856    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tzknv_de0c6b90-a1ab-467f-ae6d-c9b435756071/discovery/0.log"
Sep 18 19:17:05.995632 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:05.995601    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-6lzls_cf2c4836-df42-44a7-a965-3ec85386f45e/kube-auth-proxy/0.log"
Sep 18 19:17:06.053482 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:06.053441    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/0.log"
Sep 18 19:17:06.059259 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:06.059216    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/1.log"
Sep 18 19:17:06.104253 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:06.104216    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fshmm_9afc7b95-1779-4d18-b608-b99bca4388a0/manager/0.log"
Sep 18 19:17:06.269534 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:06.269440    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bcff78dd-rjx49_60843eb3-bc5e-4be0-8b85-0e3883447d78/manager/0.log"
Sep 18 19:17:06.286598 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:06.286571    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bcff78dd-rjx49_60843eb3-bc5e-4be0-8b85-0e3883447d78/copy-manifests/0.log"
Sep 18 19:17:08.286400 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:08.286363    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tzknv_de0c6b90-a1ab-467f-ae6d-c9b435756071/discovery/0.log"
Sep 18 19:17:08.315143 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:08.315103    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-6lzls_cf2c4836-df42-44a7-a965-3ec85386f45e/kube-auth-proxy/0.log"
Sep 18 19:17:08.370259 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:08.370224    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/0.log"
Sep 18 19:17:08.374056 ip-10-0-132-229 kubenswrapper[2558]: I0918 19:17:08.374019    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qv5bq_711f02f9-dc52-4f11-99a8-a3a5e54738b9/istio-proxy/1.log"