Sep 17 21:39:17.563345 ip-10-0-131-170 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 21:39:17.563359 ip-10-0-131-170 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 21:39:17.563368 ip-10-0-131-170 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 21:39:17.563694 ip-10-0-131-170 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 21:39:27.720179 ip-10-0-131-170 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 21:39:27.720195 ip-10-0-131-170 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2ff91f20b5e24fcd8ef12b448015f74e --
Sep 17 21:41:35.175567 ip-10-0-131-170 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 21:41:38.596839 ip-10-0-131-170 kubenswrapper[2587]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:41:38.596839 ip-10-0-131-170 kubenswrapper[2587]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 21:41:38.596839 ip-10-0-131-170 kubenswrapper[2587]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:41:38.596839 ip-10-0-131-170 kubenswrapper[2587]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 21:41:38.596839 ip-10-0-131-170 kubenswrapper[2587]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:41:38.597524 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.596919    2587 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 21:41:38.602263 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602247    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.602263 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602263    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602266    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602270    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602272    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602276    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602279    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602299    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602302    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602308    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602310    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602314    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602316    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602319    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602321    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602323    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602326    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602328    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602331    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602333    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.602342 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602338    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602342    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602345    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602348    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602352    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602354    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602357    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602360    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602362    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602365    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602368    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602370    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602373    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602375    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602378    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602380    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602383    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602385    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602388    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602391    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.602787 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602395    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602397    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602400    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602402    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602404    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602408    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602411    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602414    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602417    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602419    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602422    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602425    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602427    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602430    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602433    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602436    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602438    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602441    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602445    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.603315 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602447    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602450    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602452    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602455    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602457    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602460    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602463    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602466    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602468    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602470    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602473    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602475    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602479    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602481    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602484    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602486    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602489    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602491    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602494    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602496    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.603767 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602500    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602503    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602507    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602509    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602512    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602514    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602516    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602521    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602524    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602526    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602529    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602532    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602534    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602537    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602540    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602542    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602544    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602547    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602549    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.602560    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.604231 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603270    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603277    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603296    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603299    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603303    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603307    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603310    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603313    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603316    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603319    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603322    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603324    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603327    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603329    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603332    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603334    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603337    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603339    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603342    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603345    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.604739 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603348    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603350    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603353    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603357    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603361    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603363    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603366    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603369    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603371    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603373    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603376    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603378    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603381    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603383    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603386    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603388    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603391    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603393    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603397    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.605228 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603399    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603402    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603404    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603408    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603411    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603415    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603417    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603420    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603422    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603425    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603427    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603430    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603433    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603436    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603438    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603442    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603445    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603449    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603451    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.605737 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603454    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603457    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603460    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603463    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603465    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603468    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603471    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603473    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603476    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603478    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603481    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603484    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603487    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603490    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603492    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603495    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603497    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603500    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603502    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603505    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.606202 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603508    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603510    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603513    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603515    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603517    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603520    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603523    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603526    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603528    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603531    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603534    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603536    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603539    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603541    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603543    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603546    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603548    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603551    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603553    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603556    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.606700 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603558    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603913    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603919    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603921    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603924    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603927    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603930    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603933    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603935    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603938    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603940    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603943    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603946    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603949    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603951    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603954    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603956    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603962    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603964    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.607178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603967    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603969    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603972    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603974    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603977    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603979    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603982    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603984    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603987    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603989    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603992    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603994    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603997    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.603999    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604002    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604004    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604006    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604009    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604011    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604015    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.607643 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604018    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604022    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604026    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604028    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604031    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604034    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604037    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604040    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604043    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604045    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604049    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604052    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604054    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604057    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604059    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604062    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604064    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604067    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604069    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604072    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.608169 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604074    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604077    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604079    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604082    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604084    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604086    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604089    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604091    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604094    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604097    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604099    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604102    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604105    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604108    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604110    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604112    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604115    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604117    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604120    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604123    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.608661 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604125    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604128    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604132    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604134    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604137    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604139    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604142    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604145    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604149    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604152    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604155    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604158    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604161    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604163    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604166    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604169    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604172    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604174    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604177    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.609128 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604179    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604182    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604268    2587 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604324    2587 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604332    2587 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604336    2587 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604341    2587 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604344    2587 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604348    2587 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604352    2587 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604355    2587 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604358    2587 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604362    2587 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604365    2587 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604368    2587 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604373    2587 flags.go:64] FLAG: --cgroup-root=""
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604376    2587 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604379    2587 flags.go:64] FLAG: --client-ca-file=""
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604382    2587 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604385    2587 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604390    2587 flags.go:64] FLAG: --cluster-domain=""
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604392    2587 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604396    2587 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604399    2587 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 21:41:38.609585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604402    2587 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604406    2587 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604409    2587 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604412    2587 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604415    2587 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604418    2587 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604421    2587 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604424    2587 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604429    2587 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604432    2587 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604436    2587 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604439    2587 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604441    2587 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604444    2587 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604447    2587 flags.go:64] FLAG: --enable-server="true"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604450    2587 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604454    2587 flags.go:64] FLAG: --event-burst="100"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604470    2587 flags.go:64] FLAG: --event-qps="50"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604474    2587 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604477    2587 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604481    2587 flags.go:64] FLAG: --eviction-hard=""
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604484    2587 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604487    2587 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604491    2587 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604494    2587 flags.go:64] FLAG: --eviction-soft=""
Sep 17 21:41:38.610156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604497    2587 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604500    2587 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604502    2587 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604506    2587 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604508    2587 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604511    2587 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604514    2587 flags.go:64] FLAG: --feature-gates=""
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604518    2587 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604520    2587 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604523    2587 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604526    2587 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604529    2587 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604532    2587 flags.go:64] FLAG: --help="false"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604535    2587 flags.go:64] FLAG: --hostname-override="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604538    2587 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604542    2587 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604545    2587 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604548    2587 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604551    2587 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604554    2587 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604556    2587 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604559    2587 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604562    2587 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 21:41:38.610800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604568    2587 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604571    2587 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604574    2587 flags.go:64] FLAG: --kube-reserved=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604576    2587 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604579    2587 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604582    2587 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604584    2587 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604588    2587 flags.go:64] FLAG: --lock-file=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604591    2587 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604594    2587 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604597    2587 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604602    2587 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604605    2587 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604607    2587 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604610    2587 flags.go:64] FLAG: --logging-format="text"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604613    2587 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604616    2587 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604619    2587 flags.go:64] FLAG: --manifest-url=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604622    2587 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604626    2587 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604629    2587 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604633    2587 flags.go:64] FLAG: --max-pods="110"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604636    2587 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604639    2587 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604643    2587 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 21:41:38.611385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604646    2587 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604648    2587 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604651    2587 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604654    2587 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604661    2587 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604664    2587 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604667    2587 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604669    2587 flags.go:64] FLAG: --pod-cidr=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604673    2587 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604678    2587 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604681    2587 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604684    2587 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604687    2587 flags.go:64] FLAG: --port="10250"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604690    2587 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604692    2587 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08c3dfaee35263e39"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604696    2587 flags.go:64] FLAG: --qos-reserved=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604699    2587 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604702    2587 flags.go:64] FLAG: --register-node="true"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604705    2587 flags.go:64] FLAG: --register-with-taints=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604708    2587 flags.go:64] FLAG: --registry-burst="10"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604711    2587 flags.go:64] FLAG: --registry-qps="5"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604714    2587 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604717    2587 flags.go:64] FLAG: --reserved-memory=""
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604720    2587 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604723    2587 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 21:41:38.611992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604726    2587 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604729    2587 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604731    2587 flags.go:64] FLAG: --runonce="false"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604734    2587 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604737    2587 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604740    2587 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604744    2587 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604747    2587 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604750    2587 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604753    2587 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604756    2587 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604759    2587 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604762    2587 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604765    2587 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604767    2587 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604770    2587 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604774    2587 flags.go:64] FLAG: --system-cgroups=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604777    2587 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604782    2587 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604784    2587 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604787    2587 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604791    2587 flags.go:64] FLAG: --tls-min-version=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604794    2587 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604797    2587 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604800    2587 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 21:41:38.612683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604802    2587 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604805    2587 flags.go:64] FLAG: --v="2"
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604809    2587 flags.go:64] FLAG: --version="false"
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604813    2587 flags.go:64] FLAG: --vmodule=""
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604816    2587 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.604819    2587 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604935    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604939    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604942    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604945    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604948    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604951    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604953    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604958    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604960    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604963    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604966    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604968    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604971    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604973    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604976    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.613279 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604978    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604981    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604983    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604987    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604989    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604992    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604995    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.604998    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605001    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605003    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605006    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605008    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605011    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605013    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605016    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605018    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605020    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605023    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605025    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605028    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.613820 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605030    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605034    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605038    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605040    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605045    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605048    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605050    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605053    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605055    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605058    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605061    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605063    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605066    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605068    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605071    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605073    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605076    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605079    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605081    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605084    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.614323 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605086    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605089    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605091    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605094    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605096    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605099    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605101    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605104    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605106    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605109    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605111    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605114    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605116    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605120    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605123    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605126    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605130    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605134    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605138    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.614807 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605141    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605144    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605147    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605149    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605152    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605154    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605157    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605159    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605162    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605164    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605167    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605170    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605173    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605175    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605178    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605180    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605182    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605185    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605187    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605189    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.615261 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605192    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605194    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605197    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605199    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.605202    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.605834    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.614857    2587 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.614873    2587 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.614892    2587 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.614898    2587 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614960    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614964    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614967    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614971    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614975    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.615802 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614977    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614981    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614984    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614986    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614989    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614993    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.614998    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615000    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615003    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615005    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615008    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615011    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615013    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615015    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615018    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615020    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615023    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615025    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615028    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.616166 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615031    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615033    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615036    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615038    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615041    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615051    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615055    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615059    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615061    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615064    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615067    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615069    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615072    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615075    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615078    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615080    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615083    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615086    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615088    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615091    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.616656 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615093    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615096    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615098    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615101    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615103    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615106    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615108    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615111    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615113    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615116    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615118    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615121    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615123    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615126    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615128    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615131    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615134    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615137    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615145    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615148    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.617136 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615151    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615153    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615155    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615158    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615161    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615163    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615165    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615168    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615170    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615173    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615175    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615178    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615186    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615188    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615191    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615193    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615196    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615198    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615201    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615203    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.617633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615206    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615208    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615211    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615213    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615216    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615218    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615221    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615224    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615229    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615232    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615235    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615244    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615247    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615249    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.618162 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615252    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.615257    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615435    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615444    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615447    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615450    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615453    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615456    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615459    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615461    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615464    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615466    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615469    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615472    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615474    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:41:38.618513 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615476    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615479    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615481    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615484    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615486    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615488    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615491    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615493    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615496    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615499    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615503    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615506    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615508    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615511    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615514    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615522    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615525    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615527    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615530    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615532    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:41:38.618886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615535    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615537    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615540    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615542    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615545    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615547    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615549    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615552    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615554    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615557    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615560    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615562    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615566    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615569    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615572    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615574    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615577    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615580    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615582    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:41:38.619419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615584    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615587    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615589    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615592    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615595    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615597    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615600    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615602    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615605    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615614    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615617    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615619    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615622    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615624    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615627    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615629    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615632    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615634    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615637    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615640    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:41:38.619870 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615642    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615646    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615649    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615652    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615655    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615658    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615660    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615663    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615665    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615668    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615670    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615672    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615675    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615677    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615680    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615683    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615685    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615688    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615691    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:41:38.620354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615693    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615695    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615698    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615706    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615708    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615711    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615713    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:38.615716    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.615721    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.615837    2587 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.618176    2587 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.619344    2587 server.go:1019] "Starting client certificate rotation"
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.619446    2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 21:41:38.620806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.620126    2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 21:41:38.622054 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.622040    2587 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 21:41:38.624896 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.624870    2587 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 21:41:38.631056 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.631035    2587 log.go:25] "Validated CRI v1 runtime API"
Sep 17 21:41:38.632634 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.632618    2587 log.go:25] "Validated CRI v1 image API"
Sep 17 21:41:38.633446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.633424    2587 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 21:41:38.636747 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.636729    2587 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9785bbe1-c757-4f0a-aac5-e15649c3bb64:/dev/nvme0n1p4 bcf560af-2158-460b-a98d-d8b4da76359b:/dev/nvme0n1p3]
Sep 17 21:41:38.636810 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.636747    2587 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 21:41:38.641327 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.641201    2587 manager.go:216] Machine: {Timestamp:2026-09-17 21:41:38.640740744 +0000 UTC m=+1.989474615 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115524 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c466e009f9af23f2c1405c6eebd6 SystemUUID:ec26c466-e009-f9af-23f2-c1405c6eebd6 BootID:2ff91f20-b5e2-4fcd-8ef1-2b448015f74e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e1:ce:05:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e1:ce:05:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:55:46:76:85:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 21:41:38.641327 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.641324    2587 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 21:41:38.641453 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.641395    2587 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 21:41:38.643092 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.643071    2587 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 21:41:38.643226 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.643095    2587 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-170.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 21:41:38.644459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.644447    2587 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 21:41:38.644504 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.644462    2587 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 21:41:38.644504 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.644470    2587 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 21:41:38.644504 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.644481    2587 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 21:41:38.644586 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.644547    2587 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 21:41:38.648651 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648640    2587 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 21:41:38.648757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648748    2587 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 21:41:38.648787 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648782    2587 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 21:41:38.648819 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648792    2587 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 21:41:38.648819 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648807    2587 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 21:41:38.648819 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648817    2587 kubelet.go:398] "Adding apiserver pod source"
Sep 17 21:41:38.648920 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.648825    2587 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 21:41:38.649165 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.649154    2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 21:41:38.649227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.649174    2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 21:41:38.658941 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.658901    2587 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 21:41:38.661248 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.661234    2587 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 21:41:38.661320 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.661264    2587 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 21:41:38.662368 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662355    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 21:41:38.662405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662373    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 21:41:38.662405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662380    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 21:41:38.662405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662386    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 21:41:38.662405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662394    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 21:41:38.662405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662400    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662413    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662418    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662425    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662430    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662438    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662443    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662464    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 21:41:38.662545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.662470    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 21:41:38.663247 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.663230    2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 21:41:38.665422 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.665405    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 21:41:38.665504 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.665490    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 21:41:38.666370 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.666359    2587 server.go:1268] "Started kubelet"
Sep 17 21:41:38.666724 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.666703    2587 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 21:41:38.667229 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.666855    2587 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 21:41:38.667229 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.666899    2587 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 21:41:38.667092 ip-10-0-131-170 systemd[1]: Started Kubernetes Kubelet.
Sep 17 21:41:38.701864 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.701842    2587 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 21:41:38.702680 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.702659    2587 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 21:41:38.703050 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.703020    2587 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 21:41:38.703173 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.703155    2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4jm8"
Sep 17 21:41:38.703229 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.703176    2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4jm8"
Sep 17 21:41:38.706034 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.706017    2587 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-170.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 21:41:38.708852 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.707617    2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-170.ec2.internal.18d639a8b42ab9a2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-170.ec2.internal,UID:ip-10-0-131-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-170.ec2.internal,},FirstTimestamp:2026-09-17 21:41:38.666338722 +0000 UTC m=+2.015072593,LastTimestamp:2026-09-17 21:41:38.666338722 +0000 UTC m=+2.015072593,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-170.ec2.internal,}"
Sep 17 21:41:38.708987 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.708854    2587 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 21:41:38.708987 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.708864    2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 21:41:38.709085 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.708994    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:38.709417 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709402    2587 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 21:41:38.709525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709514    2587 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 21:41:38.709628 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709405    2587 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 21:41:38.709692 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709653    2587 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 21:41:38.709692 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709662    2587 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 21:41:38.709692 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709684    2587 factory.go:55] Registering systemd factory
Sep 17 21:41:38.709817 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.709700    2587 factory.go:223] Registration of the systemd container factory successfully
Sep 17 21:41:38.710138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710114    2587 factory.go:153] Registering CRI-O factory
Sep 17 21:41:38.710138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710133    2587 factory.go:223] Registration of the crio container factory successfully
Sep 17 21:41:38.710206 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710181    2587 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 21:41:38.710206 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710197    2587 factory.go:103] Registering Raw factory
Sep 17 21:41:38.710266 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710210    2587 manager.go:1196] Started watching for new ooms in manager
Sep 17 21:41:38.710866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.710852    2587 manager.go:349] Starting recovery of all containers
Sep 17 21:41:38.717910 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.717891    2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:38.719888 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.719871    2587 manager.go:354] Recovery completed
Sep 17 21:41:38.720776 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.720759    2587 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-170.ec2.internal\" not found" node="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.723900 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.723888    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.725435 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725421    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.725500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725448    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.725500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725459    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.725975 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725959    2587 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 21:41:38.725975 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725973    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.726126 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.725974    2587 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 21:41:38.726126 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.726063    2587 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 21:41:38.727094 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727078    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.727179 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727107    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.727179 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727121    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.727970 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727955    2587 policy_none.go:49] "None policy: Start"
Sep 17 21:41:38.727970 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727972    2587 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 21:41:38.728069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.727982    2587 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 21:41:38.729752 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.729741    2587 policy_none.go:47] "Start"
Sep 17 21:41:38.759108 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.758958    2587 manager.go:337] "Starting Device Plugin manager"
Sep 17 21:41:38.759203 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.759135    2587 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 21:41:38.759203 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759149    2587 server.go:87] "Starting device plugin registration server"
Sep 17 21:41:38.759429 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759415    2587 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 21:41:38.759478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759441    2587 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 21:41:38.759534 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759515    2587 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 21:41:38.759595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759583    2587 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 21:41:38.759636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.759597    2587 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 21:41:38.760118 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.760103    2587 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 21:41:38.761654 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.760143    2587 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:38.848770 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.848713    2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 21:41:38.850856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.850185    2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 21:41:38.850856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.850237    2587 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 21:41:38.850856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.850258    2587 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 21:41:38.850856 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.850310    2587 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 21:41:38.852721 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.852704    2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:38.859650 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.859632    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.860451 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.860430    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.860541 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.860465    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.860541 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.860479    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.860541 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.860500    2587 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.868793 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.868779    2587 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.868839 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.868801    2587 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-170.ec2.internal\": node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:38.882596 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.882578    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:38.950890 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.950856    2587 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"]
Sep 17 21:41:38.951033 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.950935    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.954949 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.954933    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.955040 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.954966    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.955040 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.954980    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.957276 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.957261    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.957380 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.957369    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.957432 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.957393    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.959630 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959603    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.959630 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959631    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.959749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959647    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.959749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959650    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.959749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959667    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.959749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.959678    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.961951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.961936    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.961999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.961970    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:41:38.962696 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.962683    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:41:38.962761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.962713    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:41:38.962761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:38.962723    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:41:38.982916 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.982897    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:38.990828 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.990814    2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-170.ec2.internal\" not found" node="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:38.995256 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:38.995242    2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-170.ec2.internal\" not found" node="ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.011456 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.011434    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9817c5934fa5926d73822e024f014f2a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-170.ec2.internal\" (UID: \"9817c5934fa5926d73822e024f014f2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.011528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.011459    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.011528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.011475    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.083013 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.082986    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.112423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112344    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.112423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112406    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.112573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112435    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9817c5934fa5926d73822e024f014f2a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-170.ec2.internal\" (UID: \"9817c5934fa5926d73822e024f014f2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.112573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112453    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.112573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112481    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9817c5934fa5926d73822e024f014f2a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-170.ec2.internal\" (UID: \"9817c5934fa5926d73822e024f014f2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.112573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.112521    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ee08e16f3736919876b0ea0982e5ca6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal\" (UID: \"4ee08e16f3736919876b0ea0982e5ca6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.183767 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.183741    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.284229 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.284198    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.294386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.294367    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.299848 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.299822    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.385347 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.385254    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.485793 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.485768    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.586263 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.586232    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.618894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.618865    2587 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 21:41:39.619331 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.619029    2587 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 21:41:39.619331 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.619081    2587 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 21:41:39.686392 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.686331    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.707904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.707864    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 21:36:38 +0000 UTC" deadline="2028-04-26 18:25:45.443739047 +0000 UTC"
Sep 17 21:41:39.707904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.707895    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14084h44m5.735846278s"
Sep 17 21:41:39.786818 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.786788    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.887543 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:39.887512    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-170.ec2.internal\" not found"
Sep 17 21:41:39.901190 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.901164    2587 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:39.909457 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.909435    2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.920664 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.920647    2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 21:41:39.922612 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.922599    2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"
Sep 17 21:41:39.934978 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.934958    2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 21:41:39.985702 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:39.985668    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9817c5934fa5926d73822e024f014f2a.slice/crio-158a3d06630fe8cf9544d678517e79dd44e53d232afd9cc6a3bd4beb26e6e24d WatchSource:0}: Error finding container 158a3d06630fe8cf9544d678517e79dd44e53d232afd9cc6a3bd4beb26e6e24d: Status 404 returned error can't find the container with id 158a3d06630fe8cf9544d678517e79dd44e53d232afd9cc6a3bd4beb26e6e24d
Sep 17 21:41:39.986039 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:39.986022    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee08e16f3736919876b0ea0982e5ca6.slice/crio-2f4481a0e40fddfccd7d221bdf6234961c45a9a20f7d81e6c101d513b5e18f4b WatchSource:0}: Error finding container 2f4481a0e40fddfccd7d221bdf6234961c45a9a20f7d81e6c101d513b5e18f4b: Status 404 returned error can't find the container with id 2f4481a0e40fddfccd7d221bdf6234961c45a9a20f7d81e6c101d513b5e18f4b
Sep 17 21:41:39.991537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:39.991523    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:41:40.152995 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.152963    2587 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:40.650053 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.650002    2587 apiserver.go:52] "Watching apiserver"
Sep 17 21:41:40.659105 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.659080    2587 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 21:41:40.663611 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.663582    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-srkwc","openshift-cluster-node-tuning-operator/tuned-vzmzp","openshift-dns/node-resolver-sz4gv","openshift-image-registry/node-ca-kpg5l","openshift-multus/multus-7hn8q","openshift-multus/multus-additional-cni-plugins-n6jj7","openshift-multus/network-metrics-daemon-t5lrs","openshift-network-diagnostics/network-check-target-62j2p","openshift-ovn-kubernetes/ovnkube-node-9vm42","kube-system/global-pull-secret-syncer-srdl6","kube-system/konnectivity-agent-jq6hw","kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal"]
Sep 17 21:41:40.666450 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.666428    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:40.667118 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.667089    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:40.668616 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.668595    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.671370 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.671347    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n8xcc\""
Sep 17 21:41:40.671581 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.671563    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.671652 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.671638    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.671946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.671924    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 21:41:40.672050 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.672033    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.674320 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.674278    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.675875 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.675853    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 21:41:40.677244 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677217    2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:40.677402 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677385    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.677483 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677452    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 21:41:40.677602 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677586    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 21:41:40.677641 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677612    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7j2c4\""
Sep 17 21:41:40.677692 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677675    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.677794 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677777    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.677978 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.677955    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.678070 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.678032    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:40.679548 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.679525    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bpgs5\""
Sep 17 21:41:40.679815 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.679796    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 21:41:40.680511 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.680492    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 21:41:40.680765 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.680748    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.682550 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.682535    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.682733 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.682695    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.684972 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.684951    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.687357 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.687340    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.690725 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.690708    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.690810 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.690772    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:40.693253 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.693235    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.695812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.695796    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 21:41:40.695913 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.695870    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 21:41:40.696087 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.696072    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.696152 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.696090    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.696432 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.696322    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 21:41:40.696525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.696505    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.696886 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.696760    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.697986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.697374    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.697986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.697414    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 21:41:40.697986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.697441    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hn5gq\""
Sep 17 21:41:40.697986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.697442    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 21:41:40.697986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.697672    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.698261 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698063    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4kz2f\""
Sep 17 21:41:40.698485 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698422    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9mql7\""
Sep 17 21:41:40.698689 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698656    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.698759 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698729    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 21:41:40.698971 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698850    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 21:41:40.698971 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.698881    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xcdzn\""
Sep 17 21:41:40.699086 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.699028    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k8cqt\""
Sep 17 21:41:40.699937 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.699918    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 21:41:40.701555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.701107    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 21:41:40.701555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.701247    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j6cpg\""
Sep 17 21:41:40.709505 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.709474    2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 21:41:40.711489 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.711471    2587 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 21:41:40.722098 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722077    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-conf-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.722189 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722109    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.722189 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722137    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-registration-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.722189 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722163    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysconfig\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722189 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722185    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-sys\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722210    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-kubelet\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722231    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-etc-kubernetes\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722253    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-conf\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722275    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-host\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722325    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-systemd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722346    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-tuned\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722374    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722395    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-lib-modules\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722418    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722440    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-bin\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722468    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-netd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722489    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-script-lib\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722512    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv4mz\" (UniqueName: \"kubernetes.io/projected/261a5def-2288-4e8a-82bf-ba2a02d8482f-kube-api-access-xv4mz\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722535    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722570    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722594    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-daemon-config\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722620    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbhl5\" (UniqueName: \"kubernetes.io/projected/43badf67-82f0-414c-82d2-84bf1b7ac9a9-kube-api-access-mbhl5\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722645    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722667    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-sys-fs\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722689    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-socket-dir-parent\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722712    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-var-lib-kubelet\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722739    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69xfk\" (UniqueName: \"kubernetes.io/projected/3fcc9579-07ff-48bd-8d4c-f22878348ec8-kube-api-access-69xfk\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722761    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-kubelet\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722785    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.722865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722812    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-env-overrides\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722834    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43badf67-82f0-414c-82d2-84bf1b7ac9a9-host\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722855    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-hostroot\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722877    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-etc-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722899    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-log-socket\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722924    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdt7t\" (UniqueName: \"kubernetes.io/projected/9d2ab88e-a956-41ad-b84f-02f349dea2c2-kube-api-access-mdt7t\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722947    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-dbus\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722970    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.722997    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-host-slash\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723021    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-bin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723044    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220a0dbe-bcb0-4145-9aef-9422d1200df8-agent-certs\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723066    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-cnibin\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723087    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-cnibin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723108    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-run\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723130    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43badf67-82f0-414c-82d2-84bf1b7ac9a9-serviceca\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723153    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723180    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-socket-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.723447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723203    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-kubernetes\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723224    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723246    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-systemd\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723269    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-var-lib-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723311    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-config\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723336    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-kubelet-config\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723366    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-system-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723391    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdhrk\" (UniqueName: \"kubernetes.io/projected/db946837-1b1b-444b-bc9a-96bc06db16d3-kube-api-access-cdhrk\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723425    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h29ft\" (UniqueName: \"kubernetes.io/projected/4b057f27-b303-4666-91ae-49a4bdde3fde-kube-api-access-h29ft\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723449    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723506    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwxb\" (UniqueName: \"kubernetes.io/projected/6773c707-e352-49e5-8a3f-550d073266ee-kube-api-access-xlwxb\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723532    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-multus\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723557    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723584    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-device-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723609    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb7mk\" (UniqueName: \"kubernetes.io/projected/adbd72ad-3bf1-454d-a982-53b70b61673f-kube-api-access-lb7mk\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723636    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h2s5\" (UniqueName: \"kubernetes.io/projected/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-kube-api-access-2h2s5\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.724213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723660    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-cni-binary-copy\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723682    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-modprobe-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723709    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-node-log\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723734    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-netns\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723758    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-multus-certs\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723782    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220a0dbe-bcb0-4145-9aef-9422d1200df8-konnectivity-ca\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723816    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-systemd-units\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723841    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-iptables-alerter-script\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723866    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723889    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-ovn\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723924    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovn-node-metrics-cert\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723950    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.723978    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-k8s-cni-cncf-io\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724004    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724029    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2ab88e-a956-41ad-b84f-02f349dea2c2-hosts-file\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724056    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2ab88e-a956-41ad-b84f-02f349dea2c2-tmp-dir\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724079    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-tmp\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.724854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724104    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-slash\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.725661 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.724128    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-netns\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.729991 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.729972    2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 21:41:40.753346 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.753320    2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pthsz"
Sep 17 21:41:40.762156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.762121    2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pthsz"
Sep 17 21:41:40.824393 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824353    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdhrk\" (UniqueName: \"kubernetes.io/projected/db946837-1b1b-444b-bc9a-96bc06db16d3-kube-api-access-cdhrk\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.824393 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824390    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h29ft\" (UniqueName: \"kubernetes.io/projected/4b057f27-b303-4666-91ae-49a4bdde3fde-kube-api-access-h29ft\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824416    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824440    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwxb\" (UniqueName: \"kubernetes.io/projected/6773c707-e352-49e5-8a3f-550d073266ee-kube-api-access-xlwxb\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824459    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-multus\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824473    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824491    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-device-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824507    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb7mk\" (UniqueName: \"kubernetes.io/projected/adbd72ad-3bf1-454d-a982-53b70b61673f-kube-api-access-lb7mk\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824522    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h2s5\" (UniqueName: \"kubernetes.io/projected/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-kube-api-access-2h2s5\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824536    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-cni-binary-copy\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824551    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-modprobe-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824566    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-node-log\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824594    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-netns\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824609    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-multus-certs\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.824618 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824623    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220a0dbe-bcb0-4145-9aef-9422d1200df8-konnectivity-ca\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824619    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-device-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824641    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-systemd-units\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824659    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824675    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-systemd-units\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824679    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-iptables-alerter-script\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824694    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-multus\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824709    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824726    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-node-log\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824732    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-ovn\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.824758    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824868    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-netns\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.824880    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:41.324809359 +0000 UTC m=+4.673543217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824763    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovn-node-metrics-cert\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824933    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824963    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-k8s-cni-cncf-io\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824986    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.825399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825009    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2ab88e-a956-41ad-b84f-02f349dea2c2-hosts-file\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825032    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2ab88e-a956-41ad-b84f-02f349dea2c2-tmp-dir\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825054    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-tmp\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825077    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-slash\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825099    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-netns\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.824900    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-multus-certs\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825146    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-conf-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825196    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-modprobe-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825119    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-conf-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825323    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-cni-binary-copy\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825335    2587 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825716    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2ab88e-a956-41ad-b84f-02f349dea2c2-hosts-file\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825798    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825916    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-iptables-alerter-script\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825923    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220a0dbe-bcb0-4145-9aef-9422d1200df8-konnectivity-ca\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825982    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-slash\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.825990    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826031    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-netns\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.826177 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826060    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2ab88e-a956-41ad-b84f-02f349dea2c2-tmp-dir\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826077    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-ovn\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826117    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-run-k8s-cni-cncf-io\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826174    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826279    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826277    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-registration-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826350    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysconfig\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826373    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-sys\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826398    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-kubelet\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826419    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-etc-kubernetes\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826433    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-registration-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826440    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-conf\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826461    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-host\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826482    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-systemd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826505    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-tuned\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826495    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-sys\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826600    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-etc-kubernetes\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826625    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-host\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826662    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-systemd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826706    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826526    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826750    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-conf\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826753    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-kubelet\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826726    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysconfig\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826781    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826781    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-lib-modules\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826846    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826870    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-bin\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826912    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-netd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826933    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-script-lib\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826939    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-lib-modules\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826954    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv4mz\" (UniqueName: \"kubernetes.io/projected/261a5def-2288-4e8a-82bf-ba2a02d8482f-kube-api-access-xv4mz\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826990    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826990    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-netd\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.826966    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-cni-bin\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.827870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827012    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827054    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-daemon-config\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827083    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbhl5\" (UniqueName: \"kubernetes.io/projected/43badf67-82f0-414c-82d2-84bf1b7ac9a9-kube-api-access-mbhl5\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827105    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827113    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827162    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-sys-fs\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827188    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-socket-dir-parent\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.827210    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827211    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-var-lib-kubelet\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827238    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69xfk\" (UniqueName: \"kubernetes.io/projected/3fcc9579-07ff-48bd-8d4c-f22878348ec8-kube-api-access-69xfk\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827265    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-kubelet\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827315    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827344    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-sys-fs\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827347    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-env-overrides\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827389    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43badf67-82f0-414c-82d2-84bf1b7ac9a9-host\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.827419    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:41:41.32738694 +0000 UTC m=+4.676120816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827450    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-hostroot\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.828704 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827476    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-etc-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827519    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-log-socket\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827542    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdt7t\" (UniqueName: \"kubernetes.io/projected/9d2ab88e-a956-41ad-b84f-02f349dea2c2-kube-api-access-mdt7t\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827579    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-dbus\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827602    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827624    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-host-slash\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827660    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-bin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827694    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220a0dbe-bcb0-4145-9aef-9422d1200df8-agent-certs\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827716    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-cnibin\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827762    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-cnibin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827773    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-script-lib\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827783    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-run\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827816    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43badf67-82f0-414c-82d2-84bf1b7ac9a9-serviceca\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827848    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-env-overrides\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827875    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827424    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43badf67-82f0-414c-82d2-84bf1b7ac9a9-host\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827908    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-socket-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827950    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-daemon-config\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.829523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827969    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-kubernetes\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827997    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-var-lib-kubelet\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828024    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828079    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-host-var-lib-cni-bin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828115    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-systemd\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828133    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-sysctl-d\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828145    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-var-lib-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828171    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-config\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828202    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-kubelet-config\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828228    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-kubelet\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828274    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-hostroot\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828316    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828233    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-system-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828352    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-system-cni-dir\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828391    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-etc-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828408    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-host-slash\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828429    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828437    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-systemd\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.830362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828454    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-log-socket\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828477    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-var-lib-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828496    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-dbus\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828496    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/261a5def-2288-4e8a-82bf-ba2a02d8482f-run-openvswitch\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828541    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7896882d-59fe-4d40-a42f-6915c7355f43-kubelet-config\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.827933    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-multus-socket-dir-parent\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828765    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adbd72ad-3bf1-454d-a982-53b70b61673f-socket-dir\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828808    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-kubernetes\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828865    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db946837-1b1b-444b-bc9a-96bc06db16d3-cnibin\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828904    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6773c707-e352-49e5-8a3f-550d073266ee-cnibin\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828950    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fcc9579-07ff-48bd-8d4c-f22878348ec8-run\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828963    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovnkube-config\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.828999    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43badf67-82f0-414c-82d2-84bf1b7ac9a9-serviceca\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.829304    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6773c707-e352-49e5-8a3f-550d073266ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.829346    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-tmp\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.831192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.829448    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/261a5def-2288-4e8a-82bf-ba2a02d8482f-ovn-node-metrics-cert\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.831884 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.831262    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fcc9579-07ff-48bd-8d4c-f22878348ec8-etc-tuned\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.831884 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.831804    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220a0dbe-bcb0-4145-9aef-9422d1200df8-agent-certs\") pod \"konnectivity-agent-jq6hw\" (UID: \"220a0dbe-bcb0-4145-9aef-9422d1200df8\") " pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:40.840742 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.840719    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h2s5\" (UniqueName: \"kubernetes.io/projected/86e4eb04-f27e-4bbc-a7e4-5b098ee01761-kube-api-access-2h2s5\") pod \"iptables-alerter-srkwc\" (UID: \"86e4eb04-f27e-4bbc-a7e4-5b098ee01761\") " pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:40.841386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.841364    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h29ft\" (UniqueName: \"kubernetes.io/projected/4b057f27-b303-4666-91ae-49a4bdde3fde-kube-api-access-h29ft\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:40.842304 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.842269    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb7mk\" (UniqueName: \"kubernetes.io/projected/adbd72ad-3bf1-454d-a982-53b70b61673f-kube-api-access-lb7mk\") pod \"aws-ebs-csi-driver-node-9qf76\" (UID: \"adbd72ad-3bf1-454d-a982-53b70b61673f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.842571 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.842539    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwxb\" (UniqueName: \"kubernetes.io/projected/6773c707-e352-49e5-8a3f-550d073266ee-kube-api-access-xlwxb\") pod \"multus-additional-cni-plugins-n6jj7\" (UID: \"6773c707-e352-49e5-8a3f-550d073266ee\") " pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:40.842682 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.842591    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdhrk\" (UniqueName: \"kubernetes.io/projected/db946837-1b1b-444b-bc9a-96bc06db16d3-kube-api-access-cdhrk\") pod \"multus-7hn8q\" (UID: \"db946837-1b1b-444b-bc9a-96bc06db16d3\") " pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:40.843572 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.843259    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:40.843572 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.843279    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:40.843572 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.843355    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:40.843572 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:40.843403    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:41.34338824 +0000 UTC m=+4.692122101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:40.845044 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.845020    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv4mz\" (UniqueName: \"kubernetes.io/projected/261a5def-2288-4e8a-82bf-ba2a02d8482f-kube-api-access-xv4mz\") pod \"ovnkube-node-9vm42\" (UID: \"261a5def-2288-4e8a-82bf-ba2a02d8482f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.849784 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.849752    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdt7t\" (UniqueName: \"kubernetes.io/projected/9d2ab88e-a956-41ad-b84f-02f349dea2c2-kube-api-access-mdt7t\") pod \"node-resolver-sz4gv\" (UID: \"9d2ab88e-a956-41ad-b84f-02f349dea2c2\") " pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:40.850184 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.850071    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbhl5\" (UniqueName: \"kubernetes.io/projected/43badf67-82f0-414c-82d2-84bf1b7ac9a9-kube-api-access-mbhl5\") pod \"node-ca-kpg5l\" (UID: \"43badf67-82f0-414c-82d2-84bf1b7ac9a9\") " pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:40.856127 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.855994    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69xfk\" (UniqueName: \"kubernetes.io/projected/3fcc9579-07ff-48bd-8d4c-f22878348ec8-kube-api-access-69xfk\") pod \"tuned-vzmzp\" (UID: \"3fcc9579-07ff-48bd-8d4c-f22878348ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:40.856506 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.856462    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal" event={"ID":"4ee08e16f3736919876b0ea0982e5ca6","Type":"ContainerStarted","Data":"2f4481a0e40fddfccd7d221bdf6234961c45a9a20f7d81e6c101d513b5e18f4b"}
Sep 17 21:41:40.858085 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.857899    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal" event={"ID":"9817c5934fa5926d73822e024f014f2a","Type":"ContainerStarted","Data":"158a3d06630fe8cf9544d678517e79dd44e53d232afd9cc6a3bd4beb26e6e24d"}
Sep 17 21:41:40.981086 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.981055    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76"
Sep 17 21:41:40.990917 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:40.990803    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:41:40.993776 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:40.993746    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbd72ad_3bf1_454d_a982_53b70b61673f.slice/crio-900f43f34aadf454496a948ae109f323eecf02f2c94259e85a4c2243bdf5c167 WatchSource:0}: Error finding container 900f43f34aadf454496a948ae109f323eecf02f2c94259e85a4c2243bdf5c167: Status 404 returned error can't find the container with id 900f43f34aadf454496a948ae109f323eecf02f2c94259e85a4c2243bdf5c167
Sep 17 21:41:41.000652 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.000629    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261a5def_2288_4e8a_82bf_ba2a02d8482f.slice/crio-6fdc4d2588088d695948ad8519ca54db73ca2ddd6f0efa4d951653ff8a0410b4 WatchSource:0}: Error finding container 6fdc4d2588088d695948ad8519ca54db73ca2ddd6f0efa4d951653ff8a0410b4: Status 404 returned error can't find the container with id 6fdc4d2588088d695948ad8519ca54db73ca2ddd6f0efa4d951653ff8a0410b4
Sep 17 21:41:41.000736 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.000690    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kpg5l"
Sep 17 21:41:41.011098 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.011073    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43badf67_82f0_414c_82d2_84bf1b7ac9a9.slice/crio-4d7d57546775105ef76ade3de7ca58f7c25bedd5fa88d70f7a252d3d3e956ab5 WatchSource:0}: Error finding container 4d7d57546775105ef76ade3de7ca58f7c25bedd5fa88d70f7a252d3d3e956ab5: Status 404 returned error can't find the container with id 4d7d57546775105ef76ade3de7ca58f7c25bedd5fa88d70f7a252d3d3e956ab5
Sep 17 21:41:41.011819 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.011799    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6jj7"
Sep 17 21:41:41.019958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.019936    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-srkwc"
Sep 17 21:41:41.023865 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.023840    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6773c707_e352_49e5_8a3f_550d073266ee.slice/crio-4583fac7b2334b60086104ec04537807019412dd45caae106f010d6b0e94b0d2 WatchSource:0}: Error finding container 4583fac7b2334b60086104ec04537807019412dd45caae106f010d6b0e94b0d2: Status 404 returned error can't find the container with id 4583fac7b2334b60086104ec04537807019412dd45caae106f010d6b0e94b0d2
Sep 17 21:41:41.027155 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.027133    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hn8q"
Sep 17 21:41:41.031099 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.031071    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e4eb04_f27e_4bbc_a7e4_5b098ee01761.slice/crio-e8b259f42aa51894a4c1b874aca592a78e3557b9cb9f19b4632f19357967a2bd WatchSource:0}: Error finding container e8b259f42aa51894a4c1b874aca592a78e3557b9cb9f19b4632f19357967a2bd: Status 404 returned error can't find the container with id e8b259f42aa51894a4c1b874aca592a78e3557b9cb9f19b4632f19357967a2bd
Sep 17 21:41:41.034544 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.034522    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz4gv"
Sep 17 21:41:41.036784 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.036761    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb946837_1b1b_444b_bc9a_96bc06db16d3.slice/crio-18ce4b69ed842c6e2491edfb046cc5d5bf8ab8815191c42308d1d33539b61027 WatchSource:0}: Error finding container 18ce4b69ed842c6e2491edfb046cc5d5bf8ab8815191c42308d1d33539b61027: Status 404 returned error can't find the container with id 18ce4b69ed842c6e2491edfb046cc5d5bf8ab8815191c42308d1d33539b61027
Sep 17 21:41:41.041443 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.041425    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:41:41.044578 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:41:41.044544    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2ab88e_a956_41ad_b84f_02f349dea2c2.slice/crio-5f2e2ac2b8bbc11a89dbc1e382a4ba0ffb8721c07506bfaf53b6d60b0e8e8fd9 WatchSource:0}: Error finding container 5f2e2ac2b8bbc11a89dbc1e382a4ba0ffb8721c07506bfaf53b6d60b0e8e8fd9: Status 404 returned error can't find the container with id 5f2e2ac2b8bbc11a89dbc1e382a4ba0ffb8721c07506bfaf53b6d60b0e8e8fd9
Sep 17 21:41:41.046800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.046783    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vzmzp"
Sep 17 21:41:41.089268 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.089244    2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:41:41.331319 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.331227    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:41.331319 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.331302    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:41.331524 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.331387    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:41.331524 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.331412    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:41.331524 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.331454    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:42.331437865 +0000 UTC m=+5.680171726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:41.331524 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.331470    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:41:42.331461191 +0000 UTC m=+5.680195063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:41.431762 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.431730    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:41.431900 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.431883    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:41.431951 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.431906    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:41.431951 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.431915    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:41.432029 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.431971    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:42.431956769 +0000 UTC m=+5.780690639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:41.765375 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.765340    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 21:36:40 +0000 UTC" deadline="2028-02-27 03:17:12.341340111 +0000 UTC"
Sep 17 21:41:41.765375 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.765372    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12653h35m30.575970998s"
Sep 17 21:41:41.851500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.851456    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:41.851674 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.851587    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:41.851920 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.851880    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:41.852031 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:41.852012    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:41.861273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.861238    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hn8q" event={"ID":"db946837-1b1b-444b-bc9a-96bc06db16d3","Type":"ContainerStarted","Data":"18ce4b69ed842c6e2491edfb046cc5d5bf8ab8815191c42308d1d33539b61027"}
Sep 17 21:41:41.862921 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.862892    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76" event={"ID":"adbd72ad-3bf1-454d-a982-53b70b61673f","Type":"ContainerStarted","Data":"900f43f34aadf454496a948ae109f323eecf02f2c94259e85a4c2243bdf5c167"}
Sep 17 21:41:41.864405 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.864381    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz4gv" event={"ID":"9d2ab88e-a956-41ad-b84f-02f349dea2c2","Type":"ContainerStarted","Data":"5f2e2ac2b8bbc11a89dbc1e382a4ba0ffb8721c07506bfaf53b6d60b0e8e8fd9"}
Sep 17 21:41:41.866540 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.866515    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-srkwc" event={"ID":"86e4eb04-f27e-4bbc-a7e4-5b098ee01761","Type":"ContainerStarted","Data":"e8b259f42aa51894a4c1b874aca592a78e3557b9cb9f19b4632f19357967a2bd"}
Sep 17 21:41:41.867983 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.867962    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerStarted","Data":"4583fac7b2334b60086104ec04537807019412dd45caae106f010d6b0e94b0d2"}
Sep 17 21:41:41.869397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.869368    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kpg5l" event={"ID":"43badf67-82f0-414c-82d2-84bf1b7ac9a9","Type":"ContainerStarted","Data":"4d7d57546775105ef76ade3de7ca58f7c25bedd5fa88d70f7a252d3d3e956ab5"}
Sep 17 21:41:41.870698 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:41.870673    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"6fdc4d2588088d695948ad8519ca54db73ca2ddd6f0efa4d951653ff8a0410b4"}
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.340639    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.340701    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.340819    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.340868    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:44.340850846 +0000 UTC m=+7.689584706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.341214    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:42.341304 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.341260    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:41:44.341245885 +0000 UTC m=+7.689979746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:42.442158 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.441909    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:42.442343 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.442183    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:42.442343 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.442213    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:42.442343 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.442237    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:42.442343 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.442324    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:44.442303585 +0000 UTC m=+7.791037457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:42.765946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.765880    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 21:36:40 +0000 UTC" deadline="2028-06-18 12:09:03.563065224 +0000 UTC"
Sep 17 21:41:42.765946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.765945    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15350h27m20.797124944s"
Sep 17 21:41:42.853336 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.853273    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:42.853497 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:42.853425    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:42.880722 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.880606    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq6hw" event={"ID":"220a0dbe-bcb0-4145-9aef-9422d1200df8","Type":"ContainerStarted","Data":"a2decb369148c51d15f28882398df13cdcefa1f2c0ca686835d3e779405a12f4"}
Sep 17 21:41:42.892036 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.891154    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal" event={"ID":"9817c5934fa5926d73822e024f014f2a","Type":"ContainerStarted","Data":"289eb4ae818e3fbe42d55d7604367fc00b3b695ec95af406b3920813a18722e0"}
Sep 17 21:41:42.893885 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:42.893811    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vzmzp" event={"ID":"3fcc9579-07ff-48bd-8d4c-f22878348ec8","Type":"ContainerStarted","Data":"032ff10adc0d82339bac2572c959fb06686b81a8061852cbe3e41f987d8b50f6"}
Sep 17 21:41:43.850867 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:43.850833    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:43.851329 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:43.850999    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:43.851431 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:43.851408    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:43.851524 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:43.851503    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.360574    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.360643    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.360763    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.360774    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.360834    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:48.360816174 +0000 UTC m=+11.709550034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:44.361199 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.360864    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:41:48.360843817 +0000 UTC m=+11.709577686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:44.461510 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.461437    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:44.461667 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.461633    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:44.461667 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.461665    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:44.461780 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.461679    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:44.461780 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.461737    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:48.461718462 +0000 UTC m=+11.810452325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:44.851497 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.851464    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:44.851968 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:44.851601    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:44.906023 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.905985    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal" event={"ID":"4ee08e16f3736919876b0ea0982e5ca6","Type":"ContainerStarted","Data":"641a4d9ece5ef1e7484455497a38feee739a1ce84fbe0c5d796c4144277573e1"}
Sep 17 21:41:44.923140 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:44.923073    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-170.ec2.internal" podStartSLOduration=5.923052351 podStartE2EDuration="5.923052351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:41:42.913560917 +0000 UTC m=+6.262294798" watchObservedRunningTime="2026-09-17 21:41:44.923052351 +0000 UTC m=+8.271786232"
Sep 17 21:41:45.850799 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:45.850768    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:45.850981 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:45.850905    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:45.851275 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:45.851256    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:45.851387 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:45.851364    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:46.852425 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:46.852391    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:46.852766 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:46.852537    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:46.913894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:46.913809    2587 generic.go:356] "Generic (PLEG): container finished" podID="4ee08e16f3736919876b0ea0982e5ca6" containerID="641a4d9ece5ef1e7484455497a38feee739a1ce84fbe0c5d796c4144277573e1" exitCode=0
Sep 17 21:41:46.913894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:46.913860    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal" event={"ID":"4ee08e16f3736919876b0ea0982e5ca6","Type":"ContainerDied","Data":"641a4d9ece5ef1e7484455497a38feee739a1ce84fbe0c5d796c4144277573e1"}
Sep 17 21:41:47.850739 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:47.850694    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:47.850929 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:47.850848    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:47.850929 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:47.850877    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:47.851041 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:47.850988    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:48.395606 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:48.395570    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:48.396106 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:48.395616    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:48.396106 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.395726    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:48.396106 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.395734    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:48.396106 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.395775    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:56.395758707 +0000 UTC m=+19.744492577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:48.396106 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.395801    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:41:56.395782962 +0000 UTC m=+19.744516823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:48.496436 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:48.496400    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:48.496666 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.496563    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:48.496666 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.496581    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:48.496666 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.496592    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:48.496666 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.496653    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:41:56.496635972 +0000 UTC m=+19.845369833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:48.850806 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:48.850769    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:48.850996 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:48.850905    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:49.851212 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:49.851173    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:49.851755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:49.851190    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:49.851755 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:49.851308    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:49.851755 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:49.851375    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:50.851057 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:50.851021    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:50.851224 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:50.851165    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:51.851535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:51.851026    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:51.851535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:51.851055    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:51.851535 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:51.851165    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:51.851535 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:51.851277    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:52.854255 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:52.854224    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:52.854652 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:52.854354    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:53.850831 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:53.850795    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:53.850998 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:53.850935    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:53.850998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:53.850984    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:53.851126 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:53.851097    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:54.854394 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:54.854314    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:54.854796 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:54.854443    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:55.851031 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:55.850997    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:55.851306 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:55.851127    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:55.851306 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:55.851160    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:55.851306 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:55.851226    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:56.453810 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:56.453776    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:56.454225 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:56.453849    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:56.454225 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.453946    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:56.454225 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.453961    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:56.454225 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.454026    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:42:12.454006551 +0000 UTC m=+35.802740409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:41:56.454225 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.454046    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:12.454035685 +0000 UTC m=+35.802769543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:41:56.555122 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:56.555094    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:56.555300 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.555267    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:41:56.555300 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.555299    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:41:56.555409 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.555309    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:56.555409 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.555365    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:12.55534489 +0000 UTC m=+35.904078750 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:41:56.850950 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:56.850916    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:56.851112 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:56.851027    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:57.850888 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:57.850857    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:57.851326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:57.850893    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:57.851326 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:57.850989    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:57.851326 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:57.851134    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:58.859954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:58.859490    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:41:58.859954 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:58.859606    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:41:58.952058 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:58.952029    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal" event={"ID":"4ee08e16f3736919876b0ea0982e5ca6","Type":"ContainerStarted","Data":"0ffb10d82d263617ceabdb5e6ca9bed3696ddd04993f882df790cc7bbaa1279c"}
Sep 17 21:41:58.954851 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:58.954777    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerStarted","Data":"fca80391dcaa6b33988a239df895d82804a6e418d3c7873926d548d8d32d3492"}
Sep 17 21:41:59.003631 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.003355    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-170.ec2.internal" podStartSLOduration=20.003339216 podStartE2EDuration="20.003339216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:41:58.975369937 +0000 UTC m=+22.324103817" watchObservedRunningTime="2026-09-17 21:41:59.003339216 +0000 UTC m=+22.352073096"
Sep 17 21:41:59.851024 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.850683    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:41:59.851210 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.850687    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:41:59.851210 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:59.851115    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:41:59.851325 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:41:59.851245    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:41:59.958885 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.958857    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz4gv" event={"ID":"9d2ab88e-a956-41ad-b84f-02f349dea2c2","Type":"ContainerStarted","Data":"6f0a5abbc65cce696bac8bca6f1ea0e3838f506e9bd810e10b92cbfadc8504af"}
Sep 17 21:41:59.960271 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.960249    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="fca80391dcaa6b33988a239df895d82804a6e418d3c7873926d548d8d32d3492" exitCode=0
Sep 17 21:41:59.960410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.960329    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"fca80391dcaa6b33988a239df895d82804a6e418d3c7873926d548d8d32d3492"}
Sep 17 21:41:59.961673 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.961631    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kpg5l" event={"ID":"43badf67-82f0-414c-82d2-84bf1b7ac9a9","Type":"ContainerStarted","Data":"e50025724246c189e203be2cb193346f6922b87120b05d5ed4837399f6cf342a"}
Sep 17 21:41:59.964556 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.964541    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:41:59.964866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.964850    2587 generic.go:356] "Generic (PLEG): container finished" podID="261a5def-2288-4e8a-82bf-ba2a02d8482f" containerID="91c9e19419bc3f74ad874de843a0b9c55a1b75a0ed787315ebf068e7133e30c9" exitCode=1
Sep 17 21:41:59.964916 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.964906    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"741ff780f301b914bb8d8c120f48c0086ed232b62ffbb14c612dbcf56b4338b3"}
Sep 17 21:41:59.964961 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.964950    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"a6cc8a6002d1d2162c004c677035a82f19ccaa747537745b98f63a87dc863f05"}
Sep 17 21:41:59.965018 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.965005    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"8033888853bdef261e75329e9a97b795e0ca50f73c33e542bfb5f78b4f1ebd78"}
Sep 17 21:41:59.965070 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.965058    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"28d4fc4b90ebd63d04155ff1619f7c2204b94937d505d6026f6dd695552cf573"}
Sep 17 21:41:59.965121 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.965109    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerDied","Data":"91c9e19419bc3f74ad874de843a0b9c55a1b75a0ed787315ebf068e7133e30c9"}
Sep 17 21:41:59.965168 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.965157    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"4c5a3facb4b137e13b13cd3fe791122a296e8c754d16ca04cbd6207effa45367"}
Sep 17 21:41:59.966325 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.966308    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vzmzp" event={"ID":"3fcc9579-07ff-48bd-8d4c-f22878348ec8","Type":"ContainerStarted","Data":"5e63e99b4d11b8abff0d8bd73e255cb8f605a6bfd90bf68df71a84b3f56125d5"}
Sep 17 21:41:59.967665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.967646    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hn8q" event={"ID":"db946837-1b1b-444b-bc9a-96bc06db16d3","Type":"ContainerStarted","Data":"81990d5f3c1b58d5f79e5d10a59d9f726551e1659c63bf6b3c3a1fe1068cfa27"}
Sep 17 21:41:59.968986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.968966    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76" event={"ID":"adbd72ad-3bf1-454d-a982-53b70b61673f","Type":"ContainerStarted","Data":"0c7eba09b858afc1d4675cb7552dea4c0e25112e7ff985888281fa6668389816"}
Sep 17 21:41:59.970238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.970219    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq6hw" event={"ID":"220a0dbe-bcb0-4145-9aef-9422d1200df8","Type":"ContainerStarted","Data":"b0a67f1ae9c71d61351cc440452e37d47b3f57635ac93ce00c4b37510a45fd09"}
Sep 17 21:41:59.973947 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:41:59.973911    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sz4gv" podStartSLOduration=3.422106641 podStartE2EDuration="20.97389943s" totalImagesPullingTime="17.551792789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:41:59.973582414 +0000 UTC m=+23.322316293" watchObservedRunningTime="2026-09-17 21:41:59.97389943 +0000 UTC m=+23.322633311"
Sep 17 21:42:00.016347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.016274    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7hn8q" podStartSLOduration=2.96272607 podStartE2EDuration="21.016259208s" totalImagesPullingTime="18.053533138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:00.015774977 +0000 UTC m=+23.364508856" watchObservedRunningTime="2026-09-17 21:42:00.016259208 +0000 UTC m=+23.364993089"
Sep 17 21:42:00.038986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.038891    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jq6hw" podStartSLOduration=4.54371991 podStartE2EDuration="21.038876006s" totalImagesPullingTime="16.495156096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:00.038501143 +0000 UTC m=+23.387235026" watchObservedRunningTime="2026-09-17 21:42:00.038876006 +0000 UTC m=+23.387609886"
Sep 17 21:42:00.059243 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.059187    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kpg5l" podStartSLOduration=8.46985316 podStartE2EDuration="22.059171172s" totalImagesPullingTime="13.589318012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:00.058991602 +0000 UTC m=+23.407725487" watchObservedRunningTime="2026-09-17 21:42:00.059171172 +0000 UTC m=+23.407905051"
Sep 17 21:42:00.080065 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.080016    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vzmzp" podStartSLOduration=4.550845103 podStartE2EDuration="21.079999088s" totalImagesPullingTime="16.529153985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:00.079451004 +0000 UTC m=+23.428184884" watchObservedRunningTime="2026-09-17 21:42:00.079999088 +0000 UTC m=+23.428732968"
Sep 17 21:42:00.260975 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.260950    2587 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 21:42:00.476406 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.476366    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:42:00.477087 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.477062    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:42:00.775674 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.775542    2587 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T21:42:00.260968496Z","UUID":"d7bf99e3-3227-4539-add9-b325d4d8f502","Handler":null,"Name":"","Endpoint":""}
Sep 17 21:42:00.777407 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.777386    2587 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 21:42:00.777527 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.777421    2587 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 21:42:00.850712 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.850686    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:00.850846 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:00.850814    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:00.974677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.974639    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-srkwc" event={"ID":"86e4eb04-f27e-4bbc-a7e4-5b098ee01761","Type":"ContainerStarted","Data":"d2f4ecebbfc4d34a60036c07c13744bdd0aa112b8f94d6498c321e554113f57f"}
Sep 17 21:42:00.976980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.976953    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76" event={"ID":"adbd72ad-3bf1-454d-a982-53b70b61673f","Type":"ContainerStarted","Data":"e75d5a1d5f1107c446913814e0338b8e703757fa466fe97f6428c719821711ab"}
Sep 17 21:42:00.977793 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.977768    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:42:00.978213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.978198    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jq6hw"
Sep 17 21:42:00.993849 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:00.993800    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-srkwc" podStartSLOduration=4.444002267 podStartE2EDuration="21.993782832s" totalImagesPullingTime="17.549780565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:00.993322277 +0000 UTC m=+24.342056158" watchObservedRunningTime="2026-09-17 21:42:00.993782832 +0000 UTC m=+24.342516713"
Sep 17 21:42:01.850673 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:01.850643    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:01.850884 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:01.850777    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:01.850884 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:01.850805    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:01.850998 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:01.850913    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:01.983459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:01.983432    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:42:01.983966 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:01.983897    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"3306d3d0b3e8b32cc35a8cc049a5db70d6ad9fc9e90e1de90652ba0717c862c9"}
Sep 17 21:42:01.986598 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:01.986542    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76" event={"ID":"adbd72ad-3bf1-454d-a982-53b70b61673f","Type":"ContainerStarted","Data":"e05f710b80e677460f8c88c80ceba9e436e2dc9b9fd272491b38c831e10fff62"}
Sep 17 21:42:02.851416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:02.851389    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:02.851613 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:02.851513    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:03.851272 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:03.851080    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:03.851653 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:03.851080    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:03.851653 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:03.851384    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:03.851653 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:03.851428    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:04.851422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.851384    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:04.851813 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:04.851513    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:04.996892 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.996876    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:42:04.997257 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.997198    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"af97caa9ac30ab5f38d18e1e8ab48e9ea9135817232809d705ed8619c0805fec"}
Sep 17 21:42:04.997715 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.997489    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:04.997715 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.997544    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:04.997715 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.997589    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:04.998130 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:04.997785    2587 scope.go:117] "RemoveContainer" containerID="91c9e19419bc3f74ad874de843a0b9c55a1b75a0ed787315ebf068e7133e30c9"
Sep 17 21:42:05.017250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:05.017213    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:05.022065 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:05.021812    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:05.060034 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:05.059978    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qf76" podStartSLOduration=7.347416272 podStartE2EDuration="27.059957249s" totalImagesPullingTime="19.712540977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:02.006266165 +0000 UTC m=+25.355000045" watchObservedRunningTime="2026-09-17 21:42:05.059957249 +0000 UTC m=+28.408691129"
Sep 17 21:42:05.851419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:05.851198    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:05.852056 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:05.851276    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:05.852056 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:05.851503    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:05.852056 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:05.851560    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:06.000835 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.000802    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="bec210ef6141219c3e3ed4e343add2d927bd758d5a90d8f8f42d4507ce3494e2" exitCode=0
Sep 17 21:42:06.001032 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.000870    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"bec210ef6141219c3e3ed4e343add2d927bd758d5a90d8f8f42d4507ce3494e2"}
Sep 17 21:42:06.004954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.004936    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:42:06.005310 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.005268    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" event={"ID":"261a5def-2288-4e8a-82bf-ba2a02d8482f","Type":"ContainerStarted","Data":"77b116f205351e604023820a37acb5268b2e3aa828ca650114e119b9cae2cc90"}
Sep 17 21:42:06.072179 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.072130    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42" podStartSLOduration=10.323970476 podStartE2EDuration="28.072116094s" totalImagesPullingTime="17.748145618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:06.071777622 +0000 UTC m=+29.420511527" watchObservedRunningTime="2026-09-17 21:42:06.072116094 +0000 UTC m=+29.420849973"
Sep 17 21:42:06.850494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:06.850468    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:06.850671 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:06.850649    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:07.012040 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.011954    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="bcb08f4fba940dfdbd0842f9bcb9828db244d13a6224c964d49c0fb05522d97d" exitCode=0
Sep 17 21:42:07.012450 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.012044    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"bcb08f4fba940dfdbd0842f9bcb9828db244d13a6224c964d49c0fb05522d97d"}
Sep 17 21:42:07.126737 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.126704    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-srdl6"]
Sep 17 21:42:07.126869 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.126815    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:07.126919 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:07.126897    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:07.134750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.134726    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62j2p"]
Sep 17 21:42:07.134854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.134830    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:07.134963 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:07.134937    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:07.139756 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.139735    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t5lrs"]
Sep 17 21:42:07.139835 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.139820    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:07.139939 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:07.139915    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:07.278535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.278461    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz4gv_9d2ab88e-a956-41ad-b84f-02f349dea2c2/dns-node-resolver/0.log"
Sep 17 21:42:07.849700 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:07.849678    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kpg5l_43badf67-82f0-414c-82d2-84bf1b7ac9a9/node-ca/0.log"
Sep 17 21:42:08.015999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:08.015971    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="ada35c6a42802bef2eadbf5034346a39b2f48468d5efe52afe48109aa897fb6a" exitCode=0
Sep 17 21:42:08.016422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:08.016013    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"ada35c6a42802bef2eadbf5034346a39b2f48468d5efe52afe48109aa897fb6a"}
Sep 17 21:42:08.853776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:08.853629    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:08.854008 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:08.853967    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:08.854482 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:08.854449    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:08.855520 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:08.854710    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:08.855520 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:08.854787    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:08.855520 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:08.854896    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:10.851419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:10.851389    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:10.851871 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:10.851511    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:10.851871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:10.851559    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:10.851871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:10.851564    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:10.851871 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:10.851675    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:10.851871 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:10.851725    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:12.466585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.466359    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:12.467087 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.466631    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:12.467087 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.466520    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:42:12.467087 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.466716    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:42:12.467087 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.466722    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret podName:7896882d-59fe-4d40-a42f-6915c7355f43 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:44.466702662 +0000 UTC m=+67.815436539 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret") pod "global-pull-secret-syncer-srdl6" (UID: "7896882d-59fe-4d40-a42f-6915c7355f43") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:42:12.467087 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.466749    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs podName:4b057f27-b303-4666-91ae-49a4bdde3fde nodeName:}" failed. No retries permitted until 2026-09-17 21:42:44.466738342 +0000 UTC m=+67.815472200 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs") pod "network-metrics-daemon-t5lrs" (UID: "4b057f27-b303-4666-91ae-49a4bdde3fde") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:42:12.567386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.567345    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:12.567553 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.567502    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:42:12.567553 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.567521    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:42:12.567553 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.567529    2587 projected.go:196] Error preparing data for projected volume kube-api-access-86z59 for pod openshift-network-diagnostics/network-check-target-62j2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:42:12.567686 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.567591    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59 podName:213b538f-daf3-4e70-b5bf-c2b7b1a1d959 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:44.567577497 +0000 UTC m=+67.916311374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-86z59" (UniqueName: "kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59") pod "network-check-target-62j2p" (UID: "213b538f-daf3-4e70-b5bf-c2b7b1a1d959") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:42:12.850993 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.850952    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:12.851194 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.851009    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:12.851194 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.851082    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:12.851194 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:12.850952    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:12.851394 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.851218    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:12.851394 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:12.851330    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:14.032441 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:14.032403    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="ef59f724b3f35a0257d91eec90ed0acfa24b33806ea9a4a42d8154248184da5f" exitCode=0
Sep 17 21:42:14.032816 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:14.032449    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"ef59f724b3f35a0257d91eec90ed0acfa24b33806ea9a4a42d8154248184da5f"}
Sep 17 21:42:14.851327 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:14.851298    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:14.851327 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:14.851333    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:14.851619 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:14.851301    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:14.851619 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:14.851417    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:14.851619 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:14.851480    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:14.851619 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:14.851561    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:15.038032 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:15.038000    2587 generic.go:356] "Generic (PLEG): container finished" podID="6773c707-e352-49e5-8a3f-550d073266ee" containerID="5692d4913f3ebef8370150a3257ad6140dca7b9f34f33a1b7e9d993c74eacc5f" exitCode=0
Sep 17 21:42:15.038483 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:15.038041    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerDied","Data":"5692d4913f3ebef8370150a3257ad6140dca7b9f34f33a1b7e9d993c74eacc5f"}
Sep 17 21:42:16.044137 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:16.044095    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" event={"ID":"6773c707-e352-49e5-8a3f-550d073266ee","Type":"ContainerStarted","Data":"3245826d78b25d0bb0117c87e79c44a3b643fd164787e83dbf7aca31a48986f7"}
Sep 17 21:42:16.076305 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:16.076195    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n6jj7" podStartSLOduration=11.894959716 podStartE2EDuration="37.076180242s" totalImagesPullingTime="25.181220526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:16.074672817 +0000 UTC m=+39.423406700" watchObservedRunningTime="2026-09-17 21:42:16.076180242 +0000 UTC m=+39.424914121"
Sep 17 21:42:16.851063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:16.850973    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:16.851063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:16.851006    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:16.851310 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:16.850973    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:16.851310 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:16.851089    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:16.851310 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:16.851153    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:16.851310 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:16.851267    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:18.851971 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:18.851936    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:18.852414 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:18.852037    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:18.852414 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:18.852096    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:18.852414 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:18.852158    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:18.852414 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:18.852210    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:18.852414 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:18.852325    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:20.850631 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:20.850602    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:20.850631 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:20.850630    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:20.851144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:20.850607    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:20.851144 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:20.850720    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:20.851144 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:20.850768    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:20.851144 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:20.850877    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:22.850708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:22.850681    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:22.850708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:22.850695    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:22.851239 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:22.850784    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:22.851239 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:22.850858    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:22.851239 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:22.850898    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:22.851239 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:22.850962    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:24.851493 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:24.851457    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:24.851493 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:24.851479    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:24.851993 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:24.851469    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:24.851993 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:24.851581    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:24.851993 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:24.851640    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:24.851993 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:24.851694    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:26.851100 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:26.851062    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:26.851478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:26.851062    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:26.851478 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:26.851178    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:26.851478 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:26.851259    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:26.851478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:26.851325    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:26.851478 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:26.851381    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:28.851968 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:28.851933    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:28.852427 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:28.852027    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:28.852427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:28.852123    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:28.852427 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:28.852214    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:28.852427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:28.852260    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:28.852427 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:28.852323    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:30.851186 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:30.851154    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:30.851585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:30.851154    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:30.851585 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:30.851268    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:30.851585 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:30.851342    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:30.851585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:30.851154    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:30.851585 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:30.851406    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:32.851586 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:32.851501    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:32.851586 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:32.851556    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:32.851979 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:32.851501    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:32.851979 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:32.851632    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5lrs" podUID="4b057f27-b303-4666-91ae-49a4bdde3fde"
Sep 17 21:42:32.851979 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:32.851706    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62j2p" podUID="213b538f-daf3-4e70-b5bf-c2b7b1a1d959"
Sep 17 21:42:32.851979 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:32.851785    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-srdl6" podUID="7896882d-59fe-4d40-a42f-6915c7355f43"
Sep 17 21:42:33.105745 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.105670    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-170.ec2.internal" event="NodeReady"
Sep 17 21:42:33.105907 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.105808    2587 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 21:42:33.196757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.196722    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6t4gt"]
Sep 17 21:42:33.198471 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.198455    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.198863 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.198843    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fldpp"]
Sep 17 21:42:33.200459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.200439    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.203101 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.203084    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 21:42:33.203235 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.203218    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 21:42:33.204227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.204210    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 21:42:33.204227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.204210    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vmz7f\""
Sep 17 21:42:33.204416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.204238    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 21:42:33.204416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.204246    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-csvq4\""
Sep 17 21:42:33.204416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.204214    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 21:42:33.213911 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.213886    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6t4gt"]
Sep 17 21:42:33.220095 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.220075    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fldpp"]
Sep 17 21:42:33.294992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.294961    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sf2l7"]
Sep 17 21:42:33.296937 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.296916    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.302091 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.302063    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 21:42:33.302213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.302113    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5wn52\""
Sep 17 21:42:33.302213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.302071    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 21:42:33.303268 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.303249    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 21:42:33.303776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.303762    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 21:42:33.308774 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308752    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce72014b-3d76-4b31-bdd3-4d7147f82989-config-volume\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.308866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308779    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9gw9\" (UniqueName: \"kubernetes.io/projected/47a8d994-d592-4677-ba9c-5b63e154d5cc-kube-api-access-b9gw9\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.308866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308809    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce72014b-3d76-4b31-bdd3-4d7147f82989-metrics-tls\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.308866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308849    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72014b-3d76-4b31-bdd3-4d7147f82989-tmp-dir\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.308960 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308889    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpzlk\" (UniqueName: \"kubernetes.io/projected/ce72014b-3d76-4b31-bdd3-4d7147f82989-kube-api-access-rpzlk\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.308960 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.308905    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47a8d994-d592-4677-ba9c-5b63e154d5cc-cert\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.318005 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.317981    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sf2l7"]
Sep 17 21:42:33.410118 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410030    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce72014b-3d76-4b31-bdd3-4d7147f82989-metrics-tls\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.410118 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410064    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72014b-3d76-4b31-bdd3-4d7147f82989-tmp-dir\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.410118 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410095    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpzlk\" (UniqueName: \"kubernetes.io/projected/ce72014b-3d76-4b31-bdd3-4d7147f82989-kube-api-access-rpzlk\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.410118 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410112    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47a8d994-d592-4677-ba9c-5b63e154d5cc-cert\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.410452 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410130    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-crio-socket\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.410452 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410175    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce72014b-3d76-4b31-bdd3-4d7147f82989-config-volume\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.410452 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410412    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gw9\" (UniqueName: \"kubernetes.io/projected/47a8d994-d592-4677-ba9c-5b63e154d5cc-kube-api-access-b9gw9\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.410598 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410466    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.410598 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410482    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72014b-3d76-4b31-bdd3-4d7147f82989-tmp-dir\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.410598 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410578    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-data-volume\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.410721 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410630    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.410721 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410662    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x8rj\" (UniqueName: \"kubernetes.io/projected/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-api-access-8x8rj\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.410825 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.410804    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce72014b-3d76-4b31-bdd3-4d7147f82989-config-volume\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.414102 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.414078    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce72014b-3d76-4b31-bdd3-4d7147f82989-metrics-tls\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.414191 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.414112    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47a8d994-d592-4677-ba9c-5b63e154d5cc-cert\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.437247 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.437224    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9gw9\" (UniqueName: \"kubernetes.io/projected/47a8d994-d592-4677-ba9c-5b63e154d5cc-kube-api-access-b9gw9\") pod \"ingress-canary-6t4gt\" (UID: \"47a8d994-d592-4677-ba9c-5b63e154d5cc\") " pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.437995 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.437973    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpzlk\" (UniqueName: \"kubernetes.io/projected/ce72014b-3d76-4b31-bdd3-4d7147f82989-kube-api-access-rpzlk\") pod \"dns-default-fldpp\" (UID: \"ce72014b-3d76-4b31-bdd3-4d7147f82989\") " pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.508101 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.508055    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6t4gt"
Sep 17 21:42:33.511258 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511235    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-crio-socket\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511344 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511316    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511344 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511342    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-data-volume\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511342    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-crio-socket\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511359    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511427 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511374    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x8rj\" (UniqueName: \"kubernetes.io/projected/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-api-access-8x8rj\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511739    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-data-volume\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.511836 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.511821    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.512961 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.512947    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:33.513883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.513866    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.528931 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.528903    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x8rj\" (UniqueName: \"kubernetes.io/projected/3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8-kube-api-access-8x8rj\") pod \"insights-runtime-extractor-sf2l7\" (UID: \"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8\") " pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.605798 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.605767    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sf2l7"
Sep 17 21:42:33.719115 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.719047    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"]
Sep 17 21:42:33.720933 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.720918    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:33.726096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.726074    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 17 21:42:33.726096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.726098    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v7r4p\""
Sep 17 21:42:33.735485 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.735196    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"]
Sep 17 21:42:33.800852 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.800798    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6t4gt"]
Sep 17 21:42:33.801623 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.801583    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fldpp"]
Sep 17 21:42:33.804817 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:33.804794    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce72014b_3d76_4b31_bdd3_4d7147f82989.slice/crio-b0cb4fb55c1849d329b26aa474138ecdeb5c827e8eb00925f354ea2185926d4e WatchSource:0}: Error finding container b0cb4fb55c1849d329b26aa474138ecdeb5c827e8eb00925f354ea2185926d4e: Status 404 returned error can't find the container with id b0cb4fb55c1849d329b26aa474138ecdeb5c827e8eb00925f354ea2185926d4e
Sep 17 21:42:33.805361 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:33.805310    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a8d994_d592_4677_ba9c_5b63e154d5cc.slice/crio-63d17107c2b0e21d8f80b745dda577fabe95d23a0421fb997f949df56caa9c06 WatchSource:0}: Error finding container 63d17107c2b0e21d8f80b745dda577fabe95d23a0421fb997f949df56caa9c06: Status 404 returned error can't find the container with id 63d17107c2b0e21d8f80b745dda577fabe95d23a0421fb997f949df56caa9c06
Sep 17 21:42:33.877356 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.877328    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sf2l7"]
Sep 17 21:42:33.880340 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:33.880314    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0f8fc4_c81e_42a7_b6c0_ee76cbe08be8.slice/crio-24c06c641fb226673f6b9c33d64c1e60ce42e37e9ee5fd737d397e0d0996f55d WatchSource:0}: Error finding container 24c06c641fb226673f6b9c33d64c1e60ce42e37e9ee5fd737d397e0d0996f55d: Status 404 returned error can't find the container with id 24c06c641fb226673f6b9c33d64c1e60ce42e37e9ee5fd737d397e0d0996f55d
Sep 17 21:42:33.913552 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:33.913516    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-q6wtx\" (UID: \"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:34.014811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.014778    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-q6wtx\" (UID: \"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:34.014970 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:34.014897    2587 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 21:42:34.014970 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:34.014959    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates podName:5841ba25-b480-4ba5-bdc3-10d4e8a5dabe nodeName:}" failed. No retries permitted until 2026-09-17 21:42:34.514941173 +0000 UTC m=+57.863675053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates") pod "prometheus-operator-admission-webhook-5c6747dc7b-q6wtx" (UID: "5841ba25-b480-4ba5-bdc3-10d4e8a5dabe") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 21:42:34.083851 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.083814    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sf2l7" event={"ID":"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8","Type":"ContainerStarted","Data":"612f3adcf2a0ee84639df4d2a8c8cc7fc98a14a403e6cc8cb40ffcb63e668a80"}
Sep 17 21:42:34.084014 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.083878    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sf2l7" event={"ID":"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8","Type":"ContainerStarted","Data":"24c06c641fb226673f6b9c33d64c1e60ce42e37e9ee5fd737d397e0d0996f55d"}
Sep 17 21:42:34.084930 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.084900    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fldpp" event={"ID":"ce72014b-3d76-4b31-bdd3-4d7147f82989","Type":"ContainerStarted","Data":"b0cb4fb55c1849d329b26aa474138ecdeb5c827e8eb00925f354ea2185926d4e"}
Sep 17 21:42:34.085900 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.085882    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6t4gt" event={"ID":"47a8d994-d592-4677-ba9c-5b63e154d5cc","Type":"ContainerStarted","Data":"63d17107c2b0e21d8f80b745dda577fabe95d23a0421fb997f949df56caa9c06"}
Sep 17 21:42:34.517967 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.517875    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-q6wtx\" (UID: \"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:34.520932 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.520895    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5841ba25-b480-4ba5-bdc3-10d4e8a5dabe-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-q6wtx\" (UID: \"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:34.629241 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.629205    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:34.853042 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.853020    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:34.853398 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.853383    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:34.853767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.853750    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:34.856777 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.856752    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 21:42:34.858553 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.858532    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44pmn\""
Sep 17 21:42:34.858679 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.858658    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 21:42:34.858764 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.858686    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 21:42:34.858764 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.858726    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 21:42:34.858764 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.858750    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7sc2s\""
Sep 17 21:42:34.938389 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:34.938360    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"]
Sep 17 21:42:34.943633 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:34.943602    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5841ba25_b480_4ba5_bdc3_10d4e8a5dabe.slice/crio-4c6eb0690217ea35a17699079708bf5cb61b7c11b4e1a8eb3e0006c33ca46ced WatchSource:0}: Error finding container 4c6eb0690217ea35a17699079708bf5cb61b7c11b4e1a8eb3e0006c33ca46ced: Status 404 returned error can't find the container with id 4c6eb0690217ea35a17699079708bf5cb61b7c11b4e1a8eb3e0006c33ca46ced
Sep 17 21:42:35.090105 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:35.090022    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx" event={"ID":"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe","Type":"ContainerStarted","Data":"4c6eb0690217ea35a17699079708bf5cb61b7c11b4e1a8eb3e0006c33ca46ced"}
Sep 17 21:42:35.092147 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:35.092121    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sf2l7" event={"ID":"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8","Type":"ContainerStarted","Data":"2a06cc02fcb5e083973ba6585228f7c3c1146d87c5c2e7bffb60946626bbfaca"}
Sep 17 21:42:36.100772 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:36.098730    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fldpp" event={"ID":"ce72014b-3d76-4b31-bdd3-4d7147f82989","Type":"ContainerStarted","Data":"eba66ee50467f070ec27b85d70344ebe838664887da3ee5259731995cdcf19ae"}
Sep 17 21:42:37.023936 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:37.023890    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9vm42"
Sep 17 21:42:38.106250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.106213    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sf2l7" event={"ID":"3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8","Type":"ContainerStarted","Data":"8c960cc015f385c06499316b1215ce8c2bc2645003eac7ba78c9a33265dea7c0"}
Sep 17 21:42:38.107725 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.107699    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6t4gt" event={"ID":"47a8d994-d592-4677-ba9c-5b63e154d5cc","Type":"ContainerStarted","Data":"e9f611ab27dd17a55e97ea1355f28665bec0548576cc415876ad371a05568810"}
Sep 17 21:42:38.109199 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.109178    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx" event={"ID":"5841ba25-b480-4ba5-bdc3-10d4e8a5dabe","Type":"ContainerStarted","Data":"5cee23b541659b74b21b818b3844ad99ce35e4e0e6f07d2a2d53771df51da6ab"}
Sep 17 21:42:38.109390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.109370    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:38.110990 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.110966    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fldpp" event={"ID":"ce72014b-3d76-4b31-bdd3-4d7147f82989","Type":"ContainerStarted","Data":"c236ff4ad015148447480bed4307962e62e6578b4fb81282b90982fbd1359fd1"}
Sep 17 21:42:38.111128 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.111106    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:38.114159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.114146    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx"
Sep 17 21:42:38.129642 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.129605    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sf2l7" podStartSLOduration=1.893416818 podStartE2EDuration="5.129593039s" totalImagesPullingTime="3.236176221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:42:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:38.129135683 +0000 UTC m=+61.477869565" watchObservedRunningTime="2026-09-17 21:42:38.129593039 +0000 UTC m=+61.478326919"
Sep 17 21:42:38.147803 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.147760    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6t4gt" podStartSLOduration=1.716933033 podStartE2EDuration="5.14774846s" totalImagesPullingTime="3.430815427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:38.146428171 +0000 UTC m=+61.495162051" watchObservedRunningTime="2026-09-17 21:42:38.14774846 +0000 UTC m=+61.496482337"
Sep 17 21:42:38.163259 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.163221    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-q6wtx" podStartSLOduration=2.868966682 podStartE2EDuration="5.16320798s" totalImagesPullingTime="2.294241298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:38.16268541 +0000 UTC m=+61.511419298" watchObservedRunningTime="2026-09-17 21:42:38.16320798 +0000 UTC m=+61.511941857"
Sep 17 21:42:38.181667 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.181626    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fldpp" podStartSLOduration=3.278831134 podStartE2EDuration="5.181612661s" totalImagesPullingTime="1.902781527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:38.180789465 +0000 UTC m=+61.529523345" watchObservedRunningTime="2026-09-17 21:42:38.181612661 +0000 UTC m=+61.530346565"
Sep 17 21:42:38.806213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.806183    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"]
Sep 17 21:42:38.808589 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.808573    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.811812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.811781    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 21:42:38.811812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.811802    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 21:42:38.813265 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.813245    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5vqjv\""
Sep 17 21:42:38.813558 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.813538    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 21:42:38.813646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.813576    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 21:42:38.813809 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.813793    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 21:42:38.821037 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.820998    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"]
Sep 17 21:42:38.849137 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.849109    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.849236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.849143    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.849236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.849200    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gmh2\" (UniqueName: \"kubernetes.io/projected/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-kube-api-access-5gmh2\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.849236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.849218    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.949581 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.949549    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gmh2\" (UniqueName: \"kubernetes.io/projected/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-kube-api-access-5gmh2\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.949783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.949588    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.949783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.949671    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.949783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.949703    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.950277 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.950243    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.952244 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.952225    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.952348 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.952229    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:38.962654 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:38.962634    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gmh2\" (UniqueName: \"kubernetes.io/projected/e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1-kube-api-access-5gmh2\") pod \"prometheus-operator-7bd5d56d6-vprm8\" (UID: \"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:39.116440 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:39.116352    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"
Sep 17 21:42:39.391611 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:39.391527    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8"]
Sep 17 21:42:39.393977 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:39.393956    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97ac2d7_9c3e_4eae_b95f_de9b2fe97ff1.slice/crio-d7424f3a746b93cf4e4d87d4ce9872059ac30376960732f46b6ad8f9c5856d7c WatchSource:0}: Error finding container d7424f3a746b93cf4e4d87d4ce9872059ac30376960732f46b6ad8f9c5856d7c: Status 404 returned error can't find the container with id d7424f3a746b93cf4e4d87d4ce9872059ac30376960732f46b6ad8f9c5856d7c
Sep 17 21:42:40.122229 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:40.122182    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8" event={"ID":"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1","Type":"ContainerStarted","Data":"d7424f3a746b93cf4e4d87d4ce9872059ac30376960732f46b6ad8f9c5856d7c"}
Sep 17 21:42:41.128006 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:41.127969    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8" event={"ID":"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1","Type":"ContainerStarted","Data":"fd2dc9be1116ed2e55010dc347720ae9303bf9e54e13f5de6cfed7905d5d890e"}
Sep 17 21:42:41.128468 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:41.128028    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8" event={"ID":"e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1","Type":"ContainerStarted","Data":"a92ce19ae742b30d9fa0a1c147ae5d7aeabe8225febf1911079aa259c2bbff40"}
Sep 17 21:42:41.149304 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:41.149233    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-vprm8" podStartSLOduration=1.912298414 podStartE2EDuration="3.149217097s" totalImagesPullingTime="1.236918683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:41.147968335 +0000 UTC m=+64.496702218" watchObservedRunningTime="2026-09-17 21:42:41.149217097 +0000 UTC m=+64.497950976"
Sep 17 21:42:43.264516 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.264481    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"]
Sep 17 21:42:43.268557 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.268539    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.272601 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.272582    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 21:42:43.272601 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.272591    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 21:42:43.273048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.273027    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fnwbr\""
Sep 17 21:42:43.278168 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.278151    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.278308 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.278192    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l7ng\" (UniqueName: \"kubernetes.io/projected/0b48fa00-e1d5-410f-ad2b-065b561197b0-kube-api-access-4l7ng\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.278308 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.278257    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b48fa00-e1d5-410f-ad2b-065b561197b0-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.278428 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.278371    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.284298 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.284243    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"]
Sep 17 21:42:43.306541 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.306494    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-w48p9"]
Sep 17 21:42:43.309897 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.309880    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.313005 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.312980    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 17 21:42:43.313097 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.313082    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 17 21:42:43.313399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.313381    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m594j\""
Sep 17 21:42:43.314377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.314363    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 17 21:42:43.316803 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.316786    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nh28r"]
Sep 17 21:42:43.324006 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.323983    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.327368 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.327352    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 21:42:43.327957 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.327938    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-w48p9"]
Sep 17 21:42:43.328397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.328369    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tv9nv\""
Sep 17 21:42:43.328397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.328385    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 21:42:43.328532 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.328382    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 21:42:43.379679 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379648    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8j5\" (UniqueName: \"kubernetes.io/projected/321d4648-7879-4cc9-8721-1c8562caba71-kube-api-access-jf8j5\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379698    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-root\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379722    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.379812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379764    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-textfile\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379791    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379829    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379849    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379884    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-wtmp\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379912    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b48fa00-e1d5-410f-ad2b-065b561197b0-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379935    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-metrics-client-ca\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.379952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.379950    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtlbn\" (UniqueName: \"kubernetes.io/projected/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-api-access-mtlbn\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380018    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380062    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380085    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-tls\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380106    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd8f947d-fa50-43f8-a69f-2088205b7e22-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380129    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l7ng\" (UniqueName: \"kubernetes.io/projected/0b48fa00-e1d5-410f-ad2b-065b561197b0-kube-api-access-4l7ng\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.380156 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380149    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.380410 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:43.380158    2587 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 17 21:42:43.380410 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:43.380220    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls podName:0b48fa00-e1d5-410f-ad2b-065b561197b0 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:43.880199374 +0000 UTC m=+67.228933237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-2kk5n" (UID: "0b48fa00-e1d5-410f-ad2b-065b561197b0") : secret "openshift-state-metrics-tls" not found
Sep 17 21:42:43.380410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380264    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-accelerators-collector-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.380410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380328    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-sys\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.380592 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.380564    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b48fa00-e1d5-410f-ad2b-065b561197b0-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.382206 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.382189    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.392029 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.392006    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l7ng\" (UniqueName: \"kubernetes.io/projected/0b48fa00-e1d5-410f-ad2b-065b561197b0-kube-api-access-4l7ng\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.481135 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481098    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-sys\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481135 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481133    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8j5\" (UniqueName: \"kubernetes.io/projected/321d4648-7879-4cc9-8721-1c8562caba71-kube-api-access-jf8j5\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481223    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-sys\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481240    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-root\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481274    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-textfile\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481326    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.481347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481333    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-root\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481378    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481404    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.481543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481418    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-wtmp\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481441    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-metrics-client-ca\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481547    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtlbn\" (UniqueName: \"kubernetes.io/projected/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-api-access-mtlbn\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481587    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-wtmp\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481591    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481657    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-tls\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481670    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-textfile\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.481728 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:43.481687    2587 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 17 21:42:43.482000 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:43.481769    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls podName:dd8f947d-fa50-43f8-a69f-2088205b7e22 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:43.981748359 +0000 UTC m=+67.330482219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-w48p9" (UID: "dd8f947d-fa50-43f8-a69f-2088205b7e22") : secret "kube-state-metrics-tls" not found
Sep 17 21:42:43.482000 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481690    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd8f947d-fa50-43f8-a69f-2088205b7e22-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.482000 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481828    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.482000 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.481866    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-accelerators-collector-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.482203 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.482028    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd8f947d-fa50-43f8-a69f-2088205b7e22-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.482203 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.482055    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-metrics-client-ca\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.482398 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.482373    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-accelerators-collector-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.482676 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.482653    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.482783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.482760    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.484048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.484023    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.484163 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.484079    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.484262 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.484246    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321d4648-7879-4cc9-8721-1c8562caba71-node-exporter-tls\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.493462 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.493440    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8j5\" (UniqueName: \"kubernetes.io/projected/321d4648-7879-4cc9-8721-1c8562caba71-kube-api-access-jf8j5\") pod \"node-exporter-nh28r\" (UID: \"321d4648-7879-4cc9-8721-1c8562caba71\") " pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.495856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.495834    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtlbn\" (UniqueName: \"kubernetes.io/projected/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-api-access-mtlbn\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.633214 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.633121    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nh28r"
Sep 17 21:42:43.666124 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:43.666095    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321d4648_7879_4cc9_8721_1c8562caba71.slice/crio-a02401061663b77df80c7b97bf2a112ceccce917c169b5cce2fe1d4fc14c2e19 WatchSource:0}: Error finding container a02401061663b77df80c7b97bf2a112ceccce917c169b5cce2fe1d4fc14c2e19: Status 404 returned error can't find the container with id a02401061663b77df80c7b97bf2a112ceccce917c169b5cce2fe1d4fc14c2e19
Sep 17 21:42:43.885108 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.885018    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.887533 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.887511    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b48fa00-e1d5-410f-ad2b-065b561197b0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-2kk5n\" (UID: \"0b48fa00-e1d5-410f-ad2b-065b561197b0\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:43.985539 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.985502    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:43.987974 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:43.987955    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd8f947d-fa50-43f8-a69f-2088205b7e22-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-w48p9\" (UID: \"dd8f947d-fa50-43f8-a69f-2088205b7e22\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:44.138952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.138864    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nh28r" event={"ID":"321d4648-7879-4cc9-8721-1c8562caba71","Type":"ContainerStarted","Data":"a02401061663b77df80c7b97bf2a112ceccce917c169b5cce2fe1d4fc14c2e19"}
Sep 17 21:42:44.176881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.176846    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"
Sep 17 21:42:44.220932 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.220898    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9"
Sep 17 21:42:44.277740 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.277698    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:42:44.307188 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.307105    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.310464 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.310437    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.312140    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.312588    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.312843    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.313139    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fpvv4\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.314017    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.314326    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.314534    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 21:42:44.314865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.314796    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 21:42:44.315325 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.314934    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 21:42:44.315325 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.315065    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 21:42:44.389144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389052    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389132    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389254    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2g2w\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389308    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389337    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389364    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389398    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389434    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389467    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389500    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389522    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389548    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.389788 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.389596    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.485019 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.484982    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n"]
Sep 17 21:42:44.488054 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:44.488014    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b48fa00_e1d5_410f_ad2b_065b561197b0.slice/crio-1641497706fd22128222fc0327eeffb19a34cb35fe1cc0dca775e301eac74db8 WatchSource:0}: Error finding container 1641497706fd22128222fc0327eeffb19a34cb35fe1cc0dca775e301eac74db8: Status 404 returned error can't find the container with id 1641497706fd22128222fc0327eeffb19a34cb35fe1cc0dca775e301eac74db8
Sep 17 21:42:44.490270 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490109    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490270 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490158    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490270 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490213    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490270 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490246    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490270 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:44.490265    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle podName:d50af830-e866-4430-bcc0-eff16598b502 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:44.990243445 +0000 UTC m=+68.338977374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d50af830-e866-4430-bcc0-eff16598b502") : configmap references non-existent config key: ca-bundle.crt
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490336    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490377    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490410    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490431    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490465    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490509    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490549    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490593    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490627    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490668    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:44.490734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.490701    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2g2w\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.491837 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.491763    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.492877 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.492736    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.494593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.494347    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.495646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.495094    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.495646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.495600    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.495816 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.495717    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.496069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.496033    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.496171 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.496147    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.496259 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.496239    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 21:42:44.496343 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.496260    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 21:42:44.496938 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.496912    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.497712 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.497684    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.498035 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.498015    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.503595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.503575    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2g2w\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.505176 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.505151    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7896882d-59fe-4d40-a42f-6915c7355f43-original-pull-secret\") pod \"global-pull-secret-syncer-srdl6\" (UID: \"7896882d-59fe-4d40-a42f-6915c7355f43\") " pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:44.505870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.505845    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b057f27-b303-4666-91ae-49a4bdde3fde-metrics-certs\") pod \"network-metrics-daemon-t5lrs\" (UID: \"4b057f27-b303-4666-91ae-49a4bdde3fde\") " pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:44.533938 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.533853    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-w48p9"]
Sep 17 21:42:44.544379 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:44.544346    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8f947d_fa50_43f8_a69f_2088205b7e22.slice/crio-2699afe029a5e6bfe7da7a113cf5db6103f48e85a309f16466ce5d67928b11c3 WatchSource:0}: Error finding container 2699afe029a5e6bfe7da7a113cf5db6103f48e85a309f16466ce5d67928b11c3: Status 404 returned error can't find the container with id 2699afe029a5e6bfe7da7a113cf5db6103f48e85a309f16466ce5d67928b11c3
Sep 17 21:42:44.591766 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.591736    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:44.597441 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.597416    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 21:42:44.606776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.606752    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 21:42:44.615992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.615967    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86z59\" (UniqueName: \"kubernetes.io/projected/213b538f-daf3-4e70-b5bf-c2b7b1a1d959-kube-api-access-86z59\") pod \"network-check-target-62j2p\" (UID: \"213b538f-daf3-4e70-b5bf-c2b7b1a1d959\") " pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:44.769951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.769916    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44pmn\""
Sep 17 21:42:44.776404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.776377    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5lrs"
Sep 17 21:42:44.776562 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.776474    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7sc2s\""
Sep 17 21:42:44.782636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.782598    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-srdl6"
Sep 17 21:42:44.783660 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.783623    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:44.994919 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.994897    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:44.995834 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:44.995809    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:45.119748 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.119718    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fldpp"
Sep 17 21:42:45.144297 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.144250    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9" event={"ID":"dd8f947d-fa50-43f8-a69f-2088205b7e22","Type":"ContainerStarted","Data":"2699afe029a5e6bfe7da7a113cf5db6103f48e85a309f16466ce5d67928b11c3"}
Sep 17 21:42:45.146381 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.146337    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nh28r" event={"ID":"321d4648-7879-4cc9-8721-1c8562caba71","Type":"ContainerStarted","Data":"02249dbd86babb0191aaf7d53bca5a45caa4aea58b1fa157a3da5f874d467099"}
Sep 17 21:42:45.148318 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.148267    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n" event={"ID":"0b48fa00-e1d5-410f-ad2b-065b561197b0","Type":"ContainerStarted","Data":"c2716b88ee316aa80f1e6be0e8195f964fd6cac20d1a9a86f9ad185a19609459"}
Sep 17 21:42:45.148433 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.148345    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n" event={"ID":"0b48fa00-e1d5-410f-ad2b-065b561197b0","Type":"ContainerStarted","Data":"9f74eb4fe027f4e6bbf021028a771985862b46fd644882fc7931b61c236cfc85"}
Sep 17 21:42:45.148433 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.148372    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n" event={"ID":"0b48fa00-e1d5-410f-ad2b-065b561197b0","Type":"ContainerStarted","Data":"1641497706fd22128222fc0327eeffb19a34cb35fe1cc0dca775e301eac74db8"}
Sep 17 21:42:45.224041 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.224011    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:42:45.227743 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.227685    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t5lrs"]
Sep 17 21:42:45.228178 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:45.228147    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7896882d_59fe_4d40_a42f_6915c7355f43.slice/crio-2ce15005cfae1e16d3d30b6a54901a696f5eafaff035e3e902e7ce8310ea7427 WatchSource:0}: Error finding container 2ce15005cfae1e16d3d30b6a54901a696f5eafaff035e3e902e7ce8310ea7427: Status 404 returned error can't find the container with id 2ce15005cfae1e16d3d30b6a54901a696f5eafaff035e3e902e7ce8310ea7427
Sep 17 21:42:45.228723 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:45.228696    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213b538f_daf3_4e70_b5bf_c2b7b1a1d959.slice/crio-d2fecf2b50c9d4ed84d7f4aa589712239aa0f1ec54efb1cf2b36ccb83e499515 WatchSource:0}: Error finding container d2fecf2b50c9d4ed84d7f4aa589712239aa0f1ec54efb1cf2b36ccb83e499515: Status 404 returned error can't find the container with id d2fecf2b50c9d4ed84d7f4aa589712239aa0f1ec54efb1cf2b36ccb83e499515
Sep 17 21:42:45.228798 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.228739    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-srdl6"]
Sep 17 21:42:45.230037 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.230009    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62j2p"]
Sep 17 21:42:45.230585 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:45.230565    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b057f27_b303_4666_91ae_49a4bdde3fde.slice/crio-8caa50bc2235e704dbc678c2810701e1b62a74144c35bdc342efb8efede715ae WatchSource:0}: Error finding container 8caa50bc2235e704dbc678c2810701e1b62a74144c35bdc342efb8efede715ae: Status 404 returned error can't find the container with id 8caa50bc2235e704dbc678c2810701e1b62a74144c35bdc342efb8efede715ae
Sep 17 21:42:45.542866 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:45.542691    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:42:45.544877 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:45.544846    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50af830_e866_4430_bcc0_eff16598b502.slice/crio-eff58fd3eea040dbb043720408948c017a0876e70697acc1219c9b86222882e0 WatchSource:0}: Error finding container eff58fd3eea040dbb043720408948c017a0876e70697acc1219c9b86222882e0: Status 404 returned error can't find the container with id eff58fd3eea040dbb043720408948c017a0876e70697acc1219c9b86222882e0
Sep 17 21:42:46.154004 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.153925    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62j2p" event={"ID":"213b538f-daf3-4e70-b5bf-c2b7b1a1d959","Type":"ContainerStarted","Data":"d2fecf2b50c9d4ed84d7f4aa589712239aa0f1ec54efb1cf2b36ccb83e499515"}
Sep 17 21:42:46.156105 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.156044    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"eff58fd3eea040dbb043720408948c017a0876e70697acc1219c9b86222882e0"}
Sep 17 21:42:46.157665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.157635    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-srdl6" event={"ID":"7896882d-59fe-4d40-a42f-6915c7355f43","Type":"ContainerStarted","Data":"2ce15005cfae1e16d3d30b6a54901a696f5eafaff035e3e902e7ce8310ea7427"}
Sep 17 21:42:46.160641 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.160429    2587 generic.go:356] "Generic (PLEG): container finished" podID="321d4648-7879-4cc9-8721-1c8562caba71" containerID="02249dbd86babb0191aaf7d53bca5a45caa4aea58b1fa157a3da5f874d467099" exitCode=0
Sep 17 21:42:46.160641 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.160511    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nh28r" event={"ID":"321d4648-7879-4cc9-8721-1c8562caba71","Type":"ContainerDied","Data":"02249dbd86babb0191aaf7d53bca5a45caa4aea58b1fa157a3da5f874d467099"}
Sep 17 21:42:46.163190 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.163115    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5lrs" event={"ID":"4b057f27-b303-4666-91ae-49a4bdde3fde","Type":"ContainerStarted","Data":"8caa50bc2235e704dbc678c2810701e1b62a74144c35bdc342efb8efede715ae"}
Sep 17 21:42:46.275490 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.275453    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"]
Sep 17 21:42:46.278612 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.278586    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.289016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.288948    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 21:42:46.289240 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.289220    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 21:42:46.289648 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.289626    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 21:42:46.290131 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.289884    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c40oifpf606m2\""
Sep 17 21:42:46.290264 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.290157    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 21:42:46.290563 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.290542    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 21:42:46.290658 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.290604    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w49qc\""
Sep 17 21:42:46.305874 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.305838    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306012 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.305888    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306012 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.305916    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306012 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.305976    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306012 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.306004    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hgrm\" (UniqueName: \"kubernetes.io/projected/376264b5-347d-40d4-a1fb-ca58256eba35-kube-api-access-6hgrm\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306218 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.306036    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-grpc-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306218 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.306067    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.306218 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.306123    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376264b5-347d-40d4-a1fb-ca58256eba35-metrics-client-ca\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.322883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.322799    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"]
Sep 17 21:42:46.407025 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.406940    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407025 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.406990    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hgrm\" (UniqueName: \"kubernetes.io/projected/376264b5-347d-40d4-a1fb-ca58256eba35-kube-api-access-6hgrm\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407025 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407022    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-grpc-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407309 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407048    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407309 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407079    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376264b5-347d-40d4-a1fb-ca58256eba35-metrics-client-ca\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407309 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407133    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407309 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407162    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.407309 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.407187    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.408507 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.408191    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376264b5-347d-40d4-a1fb-ca58256eba35-metrics-client-ca\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.411554 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.411392    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.411554 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.411534    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.411682 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.411534    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.413151 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.413125    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-grpc-tls\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.413251 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.413199    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.413749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.413729    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/376264b5-347d-40d4-a1fb-ca58256eba35-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.429976 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.429886    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hgrm\" (UniqueName: \"kubernetes.io/projected/376264b5-347d-40d4-a1fb-ca58256eba35-kube-api-access-6hgrm\") pod \"thanos-querier-747fb7c5cc-qqn9g\" (UID: \"376264b5-347d-40d4-a1fb-ca58256eba35\") " pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:46.595347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:46.595310    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:47.378478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:47.378387    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"]
Sep 17 21:42:47.392503 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:47.392464    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376264b5_347d_40d4_a1fb_ca58256eba35.slice/crio-5df47516b497cdb89204206b7620350d39c7d5e4119bbb169c656402eb3cc857 WatchSource:0}: Error finding container 5df47516b497cdb89204206b7620350d39c7d5e4119bbb169c656402eb3cc857: Status 404 returned error can't find the container with id 5df47516b497cdb89204206b7620350d39c7d5e4119bbb169c656402eb3cc857
Sep 17 21:42:48.036428 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.035921    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f8744fd5b-2szb4"]
Sep 17 21:42:48.040254 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.040229    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.047083 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.046915    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 21:42:48.047748 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.047383    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 21:42:48.047748 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.047548    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 21:42:48.047748 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.047614    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ncm8r\""
Sep 17 21:42:48.048060 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.047852    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 21:42:48.048912 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.048747    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-65sor6pn314v8\""
Sep 17 21:42:48.061042 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.060975    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f8744fd5b-2szb4"]
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123067    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"]
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123248    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-metrics-server-audit-profiles\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123337    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123372    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-tls\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123409    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-client-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123442    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-client-certs\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123476    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2f20beab-396f-4f3a-8791-c8f875d65750-audit-log\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.124446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.123504    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gkc\" (UniqueName: \"kubernetes.io/projected/2f20beab-396f-4f3a-8791-c8f875d65750-kube-api-access-h5gkc\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.128331 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.128268    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.132127 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.131923    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-49xzx\""
Sep 17 21:42:48.134711 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.134511    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 21:42:48.141652 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.141589    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"]
Sep 17 21:42:48.181417 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.181375    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9" event={"ID":"dd8f947d-fa50-43f8-a69f-2088205b7e22","Type":"ContainerStarted","Data":"8cdbf6a0407dab949bd32a6cda82634964fa13751f64dbd5282d6772a0e9c1bd"}
Sep 17 21:42:48.181576 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.181444    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9" event={"ID":"dd8f947d-fa50-43f8-a69f-2088205b7e22","Type":"ContainerStarted","Data":"01d4d7b46e97dd5aafbac9cc15d66889ce12ac220d06d3b4ffb072751381f097"}
Sep 17 21:42:48.181576 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.181484    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9" event={"ID":"dd8f947d-fa50-43f8-a69f-2088205b7e22","Type":"ContainerStarted","Data":"bb9f8fffebc99c59f4761fc294eccc1f8be7c3ae24bcf9eff0d8e67cd5b1896e"}
Sep 17 21:42:48.184812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.184770    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nh28r" event={"ID":"321d4648-7879-4cc9-8721-1c8562caba71","Type":"ContainerStarted","Data":"5fe64de99c635fc01761da201425656d8e4578d6e7cceec4a7f915afa93ab7b8"}
Sep 17 21:42:48.185001 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.184817    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nh28r" event={"ID":"321d4648-7879-4cc9-8721-1c8562caba71","Type":"ContainerStarted","Data":"263a484e68817fd32abfde5de240754a4ec06271e0d5b65b948a278a45d14887"}
Sep 17 21:42:48.189783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.189757    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n" event={"ID":"0b48fa00-e1d5-410f-ad2b-065b561197b0","Type":"ContainerStarted","Data":"e9caefe63e95869fbcd1b096142373c355b4f1d9c3960f59afb9d7c774f921ce"}
Sep 17 21:42:48.192798 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.192664    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5lrs" event={"ID":"4b057f27-b303-4666-91ae-49a4bdde3fde","Type":"ContainerStarted","Data":"22d685b1cb205b4dd81a6ee11e76588048c0a054283956eefa646c1e01bec970"}
Sep 17 21:42:48.192798 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.192713    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5lrs" event={"ID":"4b057f27-b303-4666-91ae-49a4bdde3fde","Type":"ContainerStarted","Data":"bae6770aeda5ff005901882e02c6ffc6335a87aa0aef34481f3a5a34a1e518fb"}
Sep 17 21:42:48.194947 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.194921    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"5df47516b497cdb89204206b7620350d39c7d5e4119bbb169c656402eb3cc857"}
Sep 17 21:42:48.198729 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.198586    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8" exitCode=0
Sep 17 21:42:48.198729 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.198636    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8"}
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224490    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-client-certs\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224552    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2f20beab-396f-4f3a-8791-c8f875d65750-audit-log\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224583    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224608    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gkc\" (UniqueName: \"kubernetes.io/projected/2f20beab-396f-4f3a-8791-c8f875d65750-kube-api-access-h5gkc\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224774    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-metrics-server-audit-profiles\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224862    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224896    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-tls\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.224955    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4srnv\" (UniqueName: \"kubernetes.io/projected/63bd4e54-04c1-4419-b9b1-46c06a15c107-kube-api-access-4srnv\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.225459 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.225014    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-client-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.227790 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.226793    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-metrics-server-audit-profiles\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.232213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.228576    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f20beab-396f-4f3a-8791-c8f875d65750-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.232213 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.229276    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2f20beab-396f-4f3a-8791-c8f875d65750-audit-log\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.235593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.232700    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-client-ca-bundle\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.238835 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.238792    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-tls\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.239371 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.239304    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-w48p9" podStartSLOduration=2.395304909 podStartE2EDuration="5.239269991s" totalImagesPullingTime="2.843965082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:48.206958898 +0000 UTC m=+71.555692777" watchObservedRunningTime="2026-09-17 21:42:48.239269991 +0000 UTC m=+71.588003871"
Sep 17 21:42:48.239809 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.239769    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nh28r" podStartSLOduration=3.809358429 podStartE2EDuration="5.239756681s" totalImagesPullingTime="1.356407095s" totalInitContainerRuntime="73.991157ms" isStatefulPod=true podCreationTimestamp="2026-09-17 21:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:48.239577836 +0000 UTC m=+71.588311716" watchObservedRunningTime="2026-09-17 21:42:48.239756681 +0000 UTC m=+71.588490563"
Sep 17 21:42:48.240368 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.240349    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2f20beab-396f-4f3a-8791-c8f875d65750-secret-metrics-server-client-certs\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.255501 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.255458    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gkc\" (UniqueName: \"kubernetes.io/projected/2f20beab-396f-4f3a-8791-c8f875d65750-kube-api-access-h5gkc\") pod \"metrics-server-5f8744fd5b-2szb4\" (UID: \"2f20beab-396f-4f3a-8791-c8f875d65750\") " pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.325671 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.325587    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.325838 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.325679    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4srnv\" (UniqueName: \"kubernetes.io/projected/63bd4e54-04c1-4419-b9b1-46c06a15c107-kube-api-access-4srnv\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.326059 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:48.326034    2587 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 17 21:42:48.326152 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:42:48.326111    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert podName:63bd4e54-04c1-4419-b9b1-46c06a15c107 nodeName:}" failed. No retries permitted until 2026-09-17 21:42:48.826088462 +0000 UTC m=+72.174822320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-rj7kn" (UID: "63bd4e54-04c1-4419-b9b1-46c06a15c107") : secret "monitoring-plugin-cert" not found
Sep 17 21:42:48.351508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.349790    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t5lrs" podStartSLOduration=67.189594212 podStartE2EDuration="1m9.349770482s" totalImagesPullingTime="2.16017627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:48.309311556 +0000 UTC m=+71.658045443" watchObservedRunningTime="2026-09-17 21:42:48.349770482 +0000 UTC m=+71.698504364"
Sep 17 21:42:48.351508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.350408    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-2kk5n" podStartSLOduration=2.949253524 podStartE2EDuration="5.350390642s" totalImagesPullingTime="2.401137118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:48.349436381 +0000 UTC m=+71.698170262" watchObservedRunningTime="2026-09-17 21:42:48.350390642 +0000 UTC m=+71.699124532"
Sep 17 21:42:48.351508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.350936    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4srnv\" (UniqueName: \"kubernetes.io/projected/63bd4e54-04c1-4419-b9b1-46c06a15c107-kube-api-access-4srnv\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.352741 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.352710    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:42:48.829457 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.829419    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.833192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.833164    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63bd4e54-04c1-4419-b9b1-46c06a15c107-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-rj7kn\" (UID: \"63bd4e54-04c1-4419-b9b1-46c06a15c107\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:48.916002 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.915962    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-sgjbq"]
Sep 17 21:42:48.919250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.919228    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:42:48.922904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.922881    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qnq5l\""
Sep 17 21:42:48.923757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.923737    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 21:42:48.924152 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.924131    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 21:42:48.930024 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.930002    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngn67\" (UniqueName: \"kubernetes.io/projected/7a9b572e-494b-42de-b511-9a3f56ae5b95-kube-api-access-ngn67\") pod \"downloads-6594f4c75-sgjbq\" (UID: \"7a9b572e-494b-42de-b511-9a3f56ae5b95\") " pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:42:48.937944 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:48.937922    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-sgjbq"]
Sep 17 21:42:49.030720 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.030684    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngn67\" (UniqueName: \"kubernetes.io/projected/7a9b572e-494b-42de-b511-9a3f56ae5b95-kube-api-access-ngn67\") pod \"downloads-6594f4c75-sgjbq\" (UID: \"7a9b572e-494b-42de-b511-9a3f56ae5b95\") " pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:42:49.040438 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.040413    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngn67\" (UniqueName: \"kubernetes.io/projected/7a9b572e-494b-42de-b511-9a3f56ae5b95-kube-api-access-ngn67\") pod \"downloads-6594f4c75-sgjbq\" (UID: \"7a9b572e-494b-42de-b511-9a3f56ae5b95\") " pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:42:49.042108 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.042084    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:49.229992 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.229959    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:42:49.870107 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.870075    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:42:49.874227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.874203    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.879224 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.879204    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 17 21:42:49.879532 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.879208    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 17 21:42:49.880415 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.880394    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 17 21:42:49.880998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.880981    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2xw8t\""
Sep 17 21:42:49.881092 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881065    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 17 21:42:49.881206 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.880985    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 17 21:42:49.881277 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881228    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 17 21:42:49.881452 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881435    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 17 21:42:49.881615 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881600    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 17 21:42:49.881691 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881674    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 17 21:42:49.881791 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881777    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 17 21:42:49.881865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881828    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 17 21:42:49.881923 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881831    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 17 21:42:49.881984 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881929    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 17 21:42:49.881984 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.881970    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-btf6ftqbf1jdi\""
Sep 17 21:42:49.883904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.883881    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 17 21:42:49.895477 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.895447    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:42:49.939329 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939276    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939392    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939422    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939447    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939528 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939490    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939531    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939557    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939601    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939624    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939653    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939689    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwrp\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.939755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939734    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939764    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939792    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939824    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939845    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939892    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939931    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939957    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:49.940067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:49.939983    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041139 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041088    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041139 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041141    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041176    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041203    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041249    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041278    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041327    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041349    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041382    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041405    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwrp\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041437    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041464    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041491    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041528    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041560    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041596    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041623    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041646    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.041690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041673    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.042362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.041705    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.043400 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.043372    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.044708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.044070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.044708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.044099    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.044708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.044402    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.045014 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.044986    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.045433 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.045406    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.046072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.046045    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.047390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.047079    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.047576 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.047553    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.048311 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.047932    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.048311 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.048257    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.049132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.049070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.050135 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.050106    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.050229 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.050113    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.050509 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.050488    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.051055 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.051028    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.052074 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.052043    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.052200 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.052184    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwrp\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.052478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.052463    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.053174 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.053093    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out\") pod \"prometheus-k8s-0\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:50.187617 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:50.187534    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:42:51.800878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:51.800753    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"]
Sep 17 21:42:51.807848 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:51.807765    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:42:51.808398 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:51.808368    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bd4e54_04c1_4419_b9b1_46c06a15c107.slice/crio-deb1e1b1f9344b14d0fdcccdca910a0a9563668ce77c78e8f8734c3342e2843f WatchSource:0}: Error finding container deb1e1b1f9344b14d0fdcccdca910a0a9563668ce77c78e8f8734c3342e2843f: Status 404 returned error can't find the container with id deb1e1b1f9344b14d0fdcccdca910a0a9563668ce77c78e8f8734c3342e2843f
Sep 17 21:42:51.830506 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:51.830479    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-sgjbq"]
Sep 17 21:42:51.835743 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:51.835700    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9b572e_494b_42de_b511_9a3f56ae5b95.slice/crio-27c3982f1843c0e1b9ebd248eeaa7356e132ccb4cc6a61a141253c588fda6eae WatchSource:0}: Error finding container 27c3982f1843c0e1b9ebd248eeaa7356e132ccb4cc6a61a141253c588fda6eae: Status 404 returned error can't find the container with id 27c3982f1843c0e1b9ebd248eeaa7356e132ccb4cc6a61a141253c588fda6eae
Sep 17 21:42:51.837135 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:51.837018    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f8744fd5b-2szb4"]
Sep 17 21:42:51.842476 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:42:51.842445    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f20beab_396f_4f3a_8791_c8f875d65750.slice/crio-0b21451484f055f406a5dca0386725d317d071e3246872ce68edae06282147c7 WatchSource:0}: Error finding container 0b21451484f055f406a5dca0386725d317d071e3246872ce68edae06282147c7: Status 404 returned error can't find the container with id 0b21451484f055f406a5dca0386725d317d071e3246872ce68edae06282147c7
Sep 17 21:42:52.220557 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.220527    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-sgjbq" event={"ID":"7a9b572e-494b-42de-b511-9a3f56ae5b95","Type":"ContainerStarted","Data":"27c3982f1843c0e1b9ebd248eeaa7356e132ccb4cc6a61a141253c588fda6eae"}
Sep 17 21:42:52.221968 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.221939    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4" event={"ID":"2f20beab-396f-4f3a-8791-c8f875d65750","Type":"ContainerStarted","Data":"0b21451484f055f406a5dca0386725d317d071e3246872ce68edae06282147c7"}
Sep 17 21:42:52.224542 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.224517    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62j2p" event={"ID":"213b538f-daf3-4e70-b5bf-c2b7b1a1d959","Type":"ContainerStarted","Data":"2ab87195f6250545c1cd1cb3e74f871efaab46ccb5072377d6caccc7cd1dac7d"}
Sep 17 21:42:52.224768 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.224751    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:42:52.227306 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.227182    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"52f5700476a1e243800ee59d4b1c296772e3991e0da9f9001f95dcb4cb932824"}
Sep 17 21:42:52.227306 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.227264    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"dd5df891bb57d16aad00677d56e0fde011dc6dc7d7c59fee68f1f7a03f4d4edc"}
Sep 17 21:42:52.227500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.227321    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"7bde9e3e1c392fb936ed877b417470f625828ce24e62101351b934a2c6875150"}
Sep 17 21:42:52.229386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.228960    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-srdl6" event={"ID":"7896882d-59fe-4d40-a42f-6915c7355f43","Type":"ContainerStarted","Data":"afd512e21d04a473070e020f9453a24c109f639061ddca802e6fe4d9182a6aa7"}
Sep 17 21:42:52.230404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.230381    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn" event={"ID":"63bd4e54-04c1-4419-b9b1-46c06a15c107","Type":"ContainerStarted","Data":"deb1e1b1f9344b14d0fdcccdca910a0a9563668ce77c78e8f8734c3342e2843f"}
Sep 17 21:42:52.232317 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.232269    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6" exitCode=0
Sep 17 21:42:52.232417 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.232327    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6"}
Sep 17 21:42:52.232417 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.232386    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"8e23151c03c80406b0a8da59b169b19bb917df323bfe3164f0b49eee457b8791"}
Sep 17 21:42:52.245040 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.244994    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-62j2p" podStartSLOduration=67.980011071 podStartE2EDuration="1m14.24498049s" totalImagesPullingTime="6.264969419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:41:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:52.243583275 +0000 UTC m=+75.592317156" watchObservedRunningTime="2026-09-17 21:42:52.24498049 +0000 UTC m=+75.593714395"
Sep 17 21:42:52.300365 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:52.300317    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-srdl6" podStartSLOduration=67.025838163 podStartE2EDuration="1m13.300304084s" totalImagesPullingTime="6.274465921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:52.300185889 +0000 UTC m=+75.648919770" watchObservedRunningTime="2026-09-17 21:42:52.300304084 +0000 UTC m=+75.649037956"
Sep 17 21:42:54.248996 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.248957    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn" event={"ID":"63bd4e54-04c1-4419-b9b1-46c06a15c107","Type":"ContainerStarted","Data":"2b4c93f6949ffd8637bd4d412f2e43d4ade80e96082daa0a45ee2619a17f52f5"}
Sep 17 21:42:54.250604 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.250555    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:54.255326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.254188    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4" event={"ID":"2f20beab-396f-4f3a-8791-c8f875d65750","Type":"ContainerStarted","Data":"19c935dfc1e59ec8e83a95bcb1f2f1f379b9c414635fd28ab98442ba90eba3da"}
Sep 17 21:42:54.257401 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.257138    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn"
Sep 17 21:42:54.260621 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.260526    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"d6179a6e362190c93d99c4b29d7533955bf479455716cd9004ed846bb8c0e2e1"}
Sep 17 21:42:54.264896 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.264870    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"}
Sep 17 21:42:54.322951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.322787    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-rj7kn" podStartSLOduration=4.05611117 podStartE2EDuration="6.322765945s" totalImagesPullingTime="2.266654775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:54.281242534 +0000 UTC m=+77.629976415" watchObservedRunningTime="2026-09-17 21:42:54.322765945 +0000 UTC m=+77.671499826"
Sep 17 21:42:54.369148 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:54.368671    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4" podStartSLOduration=5.082052851 podStartE2EDuration="7.368648015s" totalImagesPullingTime="2.286595164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:54.366650374 +0000 UTC m=+77.715384254" watchObservedRunningTime="2026-09-17 21:42:54.368648015 +0000 UTC m=+77.717381900"
Sep 17 21:42:55.274955 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.274917    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"6b199b1fdec83930b2a166c6cf0fc74c6b0f8ae872a4d6cde24cd797c1c96d11"}
Sep 17 21:42:55.275400 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.274982    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" event={"ID":"376264b5-347d-40d4-a1fb-ca58256eba35","Type":"ContainerStarted","Data":"c425df781ad082bd9f57691cbc90d13c0e32ad30f8eb4d481df21be1d2d9129b"}
Sep 17 21:42:55.276031 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.276003    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:55.280916 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.280887    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"}
Sep 17 21:42:55.281061 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.280958    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"}
Sep 17 21:42:55.281061 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.280998    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"}
Sep 17 21:42:55.281061 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.281036    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"}
Sep 17 21:42:55.281205 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.281075    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerStarted","Data":"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"}
Sep 17 21:42:55.308963 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:55.308897    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g" podStartSLOduration=2.936211809 podStartE2EDuration="9.308877411s" totalImagesPullingTime="6.372665602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:55.308642825 +0000 UTC m=+78.657376706" watchObservedRunningTime="2026-09-17 21:42:55.308877411 +0000 UTC m=+78.657611292"
Sep 17 21:42:56.292416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:56.292381    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-747fb7c5cc-qqn9g"
Sep 17 21:42:56.345206 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:56.345156    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.536220479 podStartE2EDuration="12.345139012s" totalImagesPullingTime="7.783764531s" totalInitContainerRuntime="25.154002ms" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:55.369718354 +0000 UTC m=+78.718452228" watchObservedRunningTime="2026-09-17 21:42:56.345139012 +0000 UTC m=+79.693872892"
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297687    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"}
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297775    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"}
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297822    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"}
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297862    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"}
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297900    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"}
Sep 17 21:42:57.298326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.297937    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerStarted","Data":"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"}
Sep 17 21:42:57.337201 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:42:57.337136    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.197549993 podStartE2EDuration="8.337118505s" totalImagesPullingTime="4.11658757s" totalInitContainerRuntime="22.980942ms" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:42:57.334207489 +0000 UTC m=+80.682941526" watchObservedRunningTime="2026-09-17 21:42:57.337118505 +0000 UTC m=+80.685852430"
Sep 17 21:43:00.188408 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:00.188361    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:43:08.353157 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:08.353123    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:43:08.353638 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:08.353222    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:43:10.363954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:10.363915    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-sgjbq" event={"ID":"7a9b572e-494b-42de-b511-9a3f56ae5b95","Type":"ContainerStarted","Data":"db16013d240f9cb7f7702be0aeeece88ed382c6fdc65a11a25e891499469f85c"}
Sep 17 21:43:10.364559 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:10.364076    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:43:10.379642 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:10.379609    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-sgjbq"
Sep 17 21:43:10.384744 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:10.384695    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-sgjbq" podStartSLOduration=4.568618977 podStartE2EDuration="22.384655813s" totalImagesPullingTime="17.816036836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:43:10.384587059 +0000 UTC m=+93.733320945" watchObservedRunningTime="2026-09-17 21:43:10.384655813 +0000 UTC m=+93.733389689"
Sep 17 21:43:23.241652 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:23.241622    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-62j2p"
Sep 17 21:43:28.358807 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:28.358777    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:43:28.362640 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:28.362609    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f8744fd5b-2szb4"
Sep 17 21:43:50.188007 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:50.187971    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:43:50.273935 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:50.273838    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:43:50.559271 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:43:50.559243    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:03.903380 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903348    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:03.903793 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903670    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="alertmanager" containerID="cri-o://e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb" gracePeriod=120
Sep 17 21:44:03.903793 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903720    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-metric" containerID="cri-o://6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06" gracePeriod=120
Sep 17 21:44:03.903918 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903767    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy" containerID="cri-o://664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f" gracePeriod=120
Sep 17 21:44:03.903918 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903785    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="prom-label-proxy" containerID="cri-o://ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d" gracePeriod=120
Sep 17 21:44:03.903918 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903818    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="config-reloader" containerID="cri-o://4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62" gracePeriod=120
Sep 17 21:44:03.903918 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:03.903746    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-web" containerID="cri-o://c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8" gracePeriod=120
Sep 17 21:44:04.601494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601456    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d" exitCode=0
Sep 17 21:44:04.601494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601482    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06" exitCode=0
Sep 17 21:44:04.601494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601488    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f" exitCode=0
Sep 17 21:44:04.601494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601494    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62" exitCode=0
Sep 17 21:44:04.601494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601500    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb" exitCode=0
Sep 17 21:44:04.601776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601526    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"}
Sep 17 21:44:04.601776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601590    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"}
Sep 17 21:44:04.601776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601617    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"}
Sep 17 21:44:04.601776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601641    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"}
Sep 17 21:44:04.601776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:04.601664    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"}
Sep 17 21:44:05.146095 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.146069    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.340016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.339976    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340018    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340034    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340057    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340090    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340113    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340152    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340196    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340217    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340257    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2g2w\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340320    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340344    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340636 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340385    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls\") pod \"d50af830-e866-4430-bcc0-eff16598b502\" (UID: \"d50af830-e866-4430-bcc0-eff16598b502\") "
Sep 17 21:44:05.340841 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.340810    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:05.341572 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.341168    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:44:05.341572 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.341516    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:05.343800 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.343725    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.343955 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.343918    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.344144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.344051    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.344408 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.344381    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume" (OuterVolumeSpecName: "config-volume") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.344520 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.344481    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:44:05.344588 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.344530    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.344649 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.344592    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w" (OuterVolumeSpecName: "kube-api-access-w2g2w") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "kube-api-access-w2g2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:44:05.345628 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.345608    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out" (OuterVolumeSpecName: "config-out") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:44:05.348254 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.348224    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.356086 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.356062    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config" (OuterVolumeSpecName: "web-config") pod "d50af830-e866-4430-bcc0-eff16598b502" (UID: "d50af830-e866-4430-bcc0-eff16598b502"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:05.441566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441518    2587 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441561    2587 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441572    2587 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-web-config\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441580    2587 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441589    2587 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441597    2587 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441606    2587 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-config-out\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441615    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d50af830-e866-4430-bcc0-eff16598b502-metrics-client-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441622    2587 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-cluster-tls-config\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441631    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2g2w\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-kube-api-access-w2g2w\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441639    2587 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d50af830-e866-4430-bcc0-eff16598b502-alertmanager-main-db\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441648    2587 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d50af830-e866-4430-bcc0-eff16598b502-tls-assets\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.441811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.441657    2587 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d50af830-e866-4430-bcc0-eff16598b502-secret-alertmanager-main-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:05.608573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.608485    2587 generic.go:356] "Generic (PLEG): container finished" podID="d50af830-e866-4430-bcc0-eff16598b502" containerID="c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8" exitCode=0
Sep 17 21:44:05.608573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.608529    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"}
Sep 17 21:44:05.608573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.608568    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.608842 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.608605    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d50af830-e866-4430-bcc0-eff16598b502","Type":"ContainerDied","Data":"eff58fd3eea040dbb043720408948c017a0876e70697acc1219c9b86222882e0"}
Sep 17 21:44:05.608842 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.608667    2587 scope.go:117] "RemoveContainer" containerID="ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"
Sep 17 21:44:05.616757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.616734    2587 scope.go:117] "RemoveContainer" containerID="6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"
Sep 17 21:44:05.624321 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.624302    2587 scope.go:117] "RemoveContainer" containerID="664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"
Sep 17 21:44:05.631085 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.631068    2587 scope.go:117] "RemoveContainer" containerID="c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"
Sep 17 21:44:05.637054 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.637032    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:05.639250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.639232    2587 scope.go:117] "RemoveContainer" containerID="4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"
Sep 17 21:44:05.640350 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.640240    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:05.646323 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.646307    2587 scope.go:117] "RemoveContainer" containerID="e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"
Sep 17 21:44:05.653320 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.653278    2587 scope.go:117] "RemoveContainer" containerID="daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8"
Sep 17 21:44:05.660118 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.660101    2587 scope.go:117] "RemoveContainer" containerID="ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"
Sep 17 21:44:05.660430 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.660410    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d\": container with ID starting with ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d not found: ID does not exist" containerID="ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"
Sep 17 21:44:05.660478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.660440    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d"} err="failed to get container status \"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d\": rpc error: code = NotFound desc = could not find container \"ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d\": container with ID starting with ad6dd3dcac798d956bf397966cca38d17965bde3fbb73bb67a81a350f0b39c7d not found: ID does not exist"
Sep 17 21:44:05.660530 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.660481    2587 scope.go:117] "RemoveContainer" containerID="6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"
Sep 17 21:44:05.660755 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.660739    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06\": container with ID starting with 6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06 not found: ID does not exist" containerID="6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"
Sep 17 21:44:05.660822 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.660760    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06"} err="failed to get container status \"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06\": rpc error: code = NotFound desc = could not find container \"6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06\": container with ID starting with 6742e42d1ea1b282dac1c9eee05af89e4d774f99d80783047891573b6baacf06 not found: ID does not exist"
Sep 17 21:44:05.660822 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.660775    2587 scope.go:117] "RemoveContainer" containerID="664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"
Sep 17 21:44:05.660993 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.660976    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f\": container with ID starting with 664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f not found: ID does not exist" containerID="664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"
Sep 17 21:44:05.661027 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661000    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f"} err="failed to get container status \"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f\": rpc error: code = NotFound desc = could not find container \"664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f\": container with ID starting with 664d3f61e374687bcafe73bcb31495415e18acfcedd4f411a528dbcfd4a1197f not found: ID does not exist"
Sep 17 21:44:05.661027 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661016    2587 scope.go:117] "RemoveContainer" containerID="c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"
Sep 17 21:44:05.661225 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.661209    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8\": container with ID starting with c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8 not found: ID does not exist" containerID="c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"
Sep 17 21:44:05.661366 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661226    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8"} err="failed to get container status \"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8\": rpc error: code = NotFound desc = could not find container \"c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8\": container with ID starting with c7bcd43098864083f9f8516df662c400280bb8f130206c7f76847d5ca24961c8 not found: ID does not exist"
Sep 17 21:44:05.661366 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661237    2587 scope.go:117] "RemoveContainer" containerID="4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"
Sep 17 21:44:05.661490 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.661476    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62\": container with ID starting with 4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62 not found: ID does not exist" containerID="4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"
Sep 17 21:44:05.661525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661492    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62"} err="failed to get container status \"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62\": rpc error: code = NotFound desc = could not find container \"4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62\": container with ID starting with 4ffc984b6553aca7208b64c286d15076068273c66def4744eb09fb19a3c8ef62 not found: ID does not exist"
Sep 17 21:44:05.661525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661503    2587 scope.go:117] "RemoveContainer" containerID="e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"
Sep 17 21:44:05.661710 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.661692    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb\": container with ID starting with e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb not found: ID does not exist" containerID="e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"
Sep 17 21:44:05.661749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661714    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb"} err="failed to get container status \"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb\": rpc error: code = NotFound desc = could not find container \"e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb\": container with ID starting with e066e19cd863d55e570dbcc759cf7a94763468641363102fe43a6e2b3b484cfb not found: ID does not exist"
Sep 17 21:44:05.661749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661725    2587 scope.go:117] "RemoveContainer" containerID="daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8"
Sep 17 21:44:05.661953 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:05.661936    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8\": container with ID starting with daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8 not found: ID does not exist" containerID="daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8"
Sep 17 21:44:05.661990 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.661956    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8"} err="failed to get container status \"daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8\": rpc error: code = NotFound desc = could not find container \"daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8\": container with ID starting with daa3173f23e475b5c6e9af9a7bf31f78e4c2cc00dfd7cf55f3687fafef820fa8 not found: ID does not exist"
Sep 17 21:44:05.674353 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674329    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:05.674628 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674617    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="config-reloader"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674630    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="config-reloader"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674639    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-web"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674644    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-web"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674667    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674672    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy"
Sep 17 21:44:05.674683 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674683    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="alertmanager"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674688    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="alertmanager"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674693    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-metric"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674698    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-metric"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674705    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="prom-label-proxy"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674709    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="prom-label-proxy"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674716    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="init-config-reloader"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674721    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="init-config-reloader"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674777    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="alertmanager"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674788    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="config-reloader"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674793    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-web"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674800    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674807    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="kube-rbac-proxy-metric"
Sep 17 21:44:05.674858 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.674812    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d50af830-e866-4430-bcc0-eff16598b502" containerName="prom-label-proxy"
Sep 17 21:44:05.678153 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.678129    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.685233 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.685213    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 21:44:05.685341 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.685263    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 21:44:05.687132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.686088    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 21:44:05.687132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.686137    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 21:44:05.687132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.686750    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 21:44:05.687132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.686789    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 21:44:05.687132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.687053    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 21:44:05.687863 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.687550    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 21:44:05.688954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.688611    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fpvv4\""
Sep 17 21:44:05.693658 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.693638    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 21:44:05.699410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.699388    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:05.845254 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845218    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845254 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845256    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845310    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845368    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845405    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vsr\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-kube-api-access-d8vsr\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845445    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845464    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845485    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-config-volume\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845502    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845548    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845587    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845635    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-config-out\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.845672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.845650    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-web-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946391 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946272    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946391 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946343    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946391 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946387    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vsr\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-kube-api-access-d8vsr\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946422    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946451    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946479    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-config-volume\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946504    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946543    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946567    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946603    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-config-out\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946628    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-web-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.946663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946663    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.947082 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946687    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.947082 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.946848    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.947191 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.947154    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.947751 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.947721    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da97ee6d-d654-473e-a638-217f97be9213-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.949734 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.949704    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da97ee6d-d654-473e-a638-217f97be9213-config-out\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.949828 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.949800    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.949885 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.949854    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-web-config\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.949885 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.949865    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.950016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.949993    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.950209 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.950187    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-config-volume\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.950500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.950258    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.950500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.950378    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da97ee6d-d654-473e-a638-217f97be9213-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.951769 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.951750    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.958122 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.958103    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vsr\" (UniqueName: \"kubernetes.io/projected/da97ee6d-d654-473e-a638-217f97be9213-kube-api-access-d8vsr\") pod \"alertmanager-main-0\" (UID: \"da97ee6d-d654-473e-a638-217f97be9213\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:05.988554 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:05.988530    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 21:44:06.284872 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:06.284840    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 21:44:06.287996 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:44:06.287953    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda97ee6d_d654_473e_a638_217f97be9213.slice/crio-47ab867fea3618597a2e05c033710e31dfd4d182ccdb429bee1441ccbc8ade20 WatchSource:0}: Error finding container 47ab867fea3618597a2e05c033710e31dfd4d182ccdb429bee1441ccbc8ade20: Status 404 returned error can't find the container with id 47ab867fea3618597a2e05c033710e31dfd4d182ccdb429bee1441ccbc8ade20
Sep 17 21:44:06.614093 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:06.614006    2587 generic.go:356] "Generic (PLEG): container finished" podID="da97ee6d-d654-473e-a638-217f97be9213" containerID="d9ed4c104532f3b42784043c03d3d2ce09a02b7ed5125c38b70e5170a1f4a343" exitCode=0
Sep 17 21:44:06.614258 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:06.614100    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerDied","Data":"d9ed4c104532f3b42784043c03d3d2ce09a02b7ed5125c38b70e5170a1f4a343"}
Sep 17 21:44:06.614258 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:06.614179    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"47ab867fea3618597a2e05c033710e31dfd4d182ccdb429bee1441ccbc8ade20"}
Sep 17 21:44:06.857454 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:06.857423    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d50af830-e866-4430-bcc0-eff16598b502" path="/var/lib/kubelet/pods/d50af830-e866-4430-bcc0-eff16598b502/volumes"
Sep 17 21:44:07.622954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.622916    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"33ed3efddf42f54caa26f8fc2b4a4d5c133cacc23bbe98ba14b87a4449e13d39"}
Sep 17 21:44:07.623337 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.622997    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"73b44a7da440a042fab892100c64c9529673a63ba3514f4e12ff473973134c7e"}
Sep 17 21:44:07.623337 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.623038    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"4cfc19ab9d279281d02276493715e6f6c7c4bf08536cce9c05fc973125b55504"}
Sep 17 21:44:07.623337 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.623082    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"c153bfd9e853f542947e773f023d912859db6d519d28112fc02867b4b5fb8eb9"}
Sep 17 21:44:07.623337 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.623118    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"30bc2a72747c59cf60a4e7fc8f729716d5d5526051288f3363d837c084070a10"}
Sep 17 21:44:07.623337 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.623152    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da97ee6d-d654-473e-a638-217f97be9213","Type":"ContainerStarted","Data":"cb2f78366470e7ad23ef1a961ee035143a469eba5f30c709e7eb2ca628700919"}
Sep 17 21:44:07.662358 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:07.662268    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.642931752 podStartE2EDuration="2.662253391s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.321639ms" isStatefulPod=false podCreationTimestamp="2026-09-17 21:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:44:07.660865934 +0000 UTC m=+151.009599827" watchObservedRunningTime="2026-09-17 21:44:07.662253391 +0000 UTC m=+151.010987282"
Sep 17 21:44:08.306450 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306414    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:08.306894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306723    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="prometheus" containerID="cri-o://3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a" gracePeriod=600
Sep 17 21:44:08.306894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306737    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy" containerID="cri-o://90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61" gracePeriod=600
Sep 17 21:44:08.306894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306758    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22" gracePeriod=600
Sep 17 21:44:08.306894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306787    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="config-reloader" containerID="cri-o://616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d" gracePeriod=600
Sep 17 21:44:08.306894 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306728    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="thanos-sidecar" containerID="cri-o://a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00" gracePeriod=600
Sep 17 21:44:08.307238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.306752    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-web" containerID="cri-o://eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e" gracePeriod=600
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633843    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22" exitCode=0
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633871    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61" exitCode=0
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633877    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00" exitCode=0
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633883    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d" exitCode=0
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633888    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a" exitCode=0
Sep 17 21:44:08.633928 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633908    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"}
Sep 17 21:44:08.634449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.633996    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"}
Sep 17 21:44:08.634449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.634024    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"}
Sep 17 21:44:08.634449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.634047    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"}
Sep 17 21:44:08.634449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:08.634070    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"}
Sep 17 21:44:09.561480 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.561455    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.575954 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.575921    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576062 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.575966    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576062 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.575994    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576062 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576030    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576063    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576136    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576174    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576205    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zwrp\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576234    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576264    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576309    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576333    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576366    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576403    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576427    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576490    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576527    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576557    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576925 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576589    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.576925 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.576621    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle\") pod \"2a488ee3-f512-4902-b609-5a8b744c5041\" (UID: \"2a488ee3-f512-4902-b609-5a8b744c5041\") "
Sep 17 21:44:09.577274 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.577246    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:44:09.577604 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.577575    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.577989 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.577964    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.578066 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.578028    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.578238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.578209    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.578522 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.578495    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.578880 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.578821    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.579058 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.578922    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.580243 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.580214    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:44:09.581025 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.580996    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.581502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.581459    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config" (OuterVolumeSpecName: "config") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.581665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.581635    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out" (OuterVolumeSpecName: "config-out") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:44:09.581665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.581632    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp" (OuterVolumeSpecName: "kube-api-access-6zwrp") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "kube-api-access-6zwrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:44:09.581832 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.581779    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:44:09.582625 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.582599    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.582776 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.582756    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.582868 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.582851    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.582949 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.582925    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.583081 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.583057    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.592851 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.592824    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config" (OuterVolumeSpecName: "web-config") pod "2a488ee3-f512-4902-b609-5a8b744c5041" (UID: "2a488ee3-f512-4902-b609-5a8b744c5041"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:44:09.643663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.643566    2587 generic.go:356] "Generic (PLEG): container finished" podID="2a488ee3-f512-4902-b609-5a8b744c5041" containerID="eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e" exitCode=0
Sep 17 21:44:09.643663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.643638    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"}
Sep 17 21:44:09.644083 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.643667    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.644083 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.643738    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2a488ee3-f512-4902-b609-5a8b744c5041","Type":"ContainerDied","Data":"8e23151c03c80406b0a8da59b169b19bb917df323bfe3164f0b49eee457b8791"}
Sep 17 21:44:09.644083 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.643793    2587 scope.go:117] "RemoveContainer" containerID="5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"
Sep 17 21:44:09.652855 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.652836    2587 scope.go:117] "RemoveContainer" containerID="90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"
Sep 17 21:44:09.661021 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.661004    2587 scope.go:117] "RemoveContainer" containerID="eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"
Sep 17 21:44:09.668662 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.668640    2587 scope.go:117] "RemoveContainer" containerID="a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"
Sep 17 21:44:09.672502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.672481    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:09.676083 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.676063    2587 scope.go:117] "RemoveContainer" containerID="616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"
Sep 17 21:44:09.677154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677141    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zwrp\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-kube-api-access-6zwrp\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677159    2587 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677169    2587 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-metrics-client-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677178    2587 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-config\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677187    2587 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677195    2587 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677204    2587 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-config-out\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677213    2587 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677222    2587 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677229    2587 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a488ee3-f512-4902-b609-5a8b744c5041-tls-assets\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677237    2587 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-grpc-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677244    2587 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677252    2587 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677260    2587 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677268    2587 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-kube-rbac-proxy\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.677397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677277    2587 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.678367 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677406    2587 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.678367 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677418    2587 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2a488ee3-f512-4902-b609-5a8b744c5041-prometheus-k8s-db\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.678367 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677427    2587 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a488ee3-f512-4902-b609-5a8b744c5041-configmap-metrics-client-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.678367 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.677435    2587 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a488ee3-f512-4902-b609-5a8b744c5041-web-config\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:44:09.680464 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.680444    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:09.683720 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.683698    2587 scope.go:117] "RemoveContainer" containerID="3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"
Sep 17 21:44:09.691398 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.691369    2587 scope.go:117] "RemoveContainer" containerID="f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6"
Sep 17 21:44:09.698195 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.698178    2587 scope.go:117] "RemoveContainer" containerID="5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"
Sep 17 21:44:09.698475 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.698456    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22\": container with ID starting with 5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22 not found: ID does not exist" containerID="5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"
Sep 17 21:44:09.698555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.698488    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22"} err="failed to get container status \"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22\": rpc error: code = NotFound desc = could not find container \"5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22\": container with ID starting with 5eca5311e795be515ec7835ccad7dc7f8a20255d3990788cb5c3aeb77bb63a22 not found: ID does not exist"
Sep 17 21:44:09.698555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.698514    2587 scope.go:117] "RemoveContainer" containerID="90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"
Sep 17 21:44:09.698810 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.698775    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61\": container with ID starting with 90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61 not found: ID does not exist" containerID="90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"
Sep 17 21:44:09.698916 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.698817    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61"} err="failed to get container status \"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61\": rpc error: code = NotFound desc = could not find container \"90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61\": container with ID starting with 90d4d9965615d06026c1d19db20838320f1cc571b07c7111a5f475fdb795ae61 not found: ID does not exist"
Sep 17 21:44:09.698916 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.698839    2587 scope.go:117] "RemoveContainer" containerID="eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"
Sep 17 21:44:09.699163 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.699136    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e\": container with ID starting with eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e not found: ID does not exist" containerID="eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"
Sep 17 21:44:09.699163 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699171    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e"} err="failed to get container status \"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e\": rpc error: code = NotFound desc = could not find container \"eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e\": container with ID starting with eba91b1391af08a8a24b0ec4a7903ce8491b74fa77e2123204906134663fba4e not found: ID does not exist"
Sep 17 21:44:09.699388 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699193    2587 scope.go:117] "RemoveContainer" containerID="a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"
Sep 17 21:44:09.699580 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.699558    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00\": container with ID starting with a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00 not found: ID does not exist" containerID="a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"
Sep 17 21:44:09.699638 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699588    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00"} err="failed to get container status \"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00\": rpc error: code = NotFound desc = could not find container \"a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00\": container with ID starting with a28075257ec0b64bf3c1fe86ae3a0d0991503f935bd0d9db8916767612cf2f00 not found: ID does not exist"
Sep 17 21:44:09.699638 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699608    2587 scope.go:117] "RemoveContainer" containerID="616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"
Sep 17 21:44:09.699882 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.699858    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d\": container with ID starting with 616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d not found: ID does not exist" containerID="616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"
Sep 17 21:44:09.699946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699886    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d"} err="failed to get container status \"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d\": rpc error: code = NotFound desc = could not find container \"616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d\": container with ID starting with 616d6532e6d41df5b026b60e5545b29ac0d567e27469e7be4504fa10afeb098d not found: ID does not exist"
Sep 17 21:44:09.699946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.699907    2587 scope.go:117] "RemoveContainer" containerID="3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"
Sep 17 21:44:09.700173 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.700157    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a\": container with ID starting with 3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a not found: ID does not exist" containerID="3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"
Sep 17 21:44:09.700212 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.700179    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a"} err="failed to get container status \"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a\": rpc error: code = NotFound desc = could not find container \"3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a\": container with ID starting with 3f88a7b3e4796d3060409de04f3cb97c8a8f64a07ef825826cd0338a674dba7a not found: ID does not exist"
Sep 17 21:44:09.700212 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.700194    2587 scope.go:117] "RemoveContainer" containerID="f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6"
Sep 17 21:44:09.700509 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:44:09.700471    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6\": container with ID starting with f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6 not found: ID does not exist" containerID="f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6"
Sep 17 21:44:09.700509 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.700493    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6"} err="failed to get container status \"f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6\": rpc error: code = NotFound desc = could not find container \"f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6\": container with ID starting with f3c3cfc45ec2b3ff5735456543e947a54f80c2ac443d7a34b0f73815207242f6 not found: ID does not exist"
Sep 17 21:44:09.710030 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710009    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:09.710324 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710311    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="init-config-reloader"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710326    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="init-config-reloader"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710342    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="thanos-sidecar"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710347    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="thanos-sidecar"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710358    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="prometheus"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710363    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="prometheus"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710369    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy"
Sep 17 21:44:09.710377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710374    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710388    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-web"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710392    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-web"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710400    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="config-reloader"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710405    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="config-reloader"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710410    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-thanos"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710415    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-thanos"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710458    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-web"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710465    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="thanos-sidecar"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710475    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy-thanos"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710480    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="prometheus"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710485    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="config-reloader"
Sep 17 21:44:09.710570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.710489    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" containerName="kube-rbac-proxy"
Sep 17 21:44:09.714086 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.714068    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.717254    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.717390    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.717641    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.718022    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.718694    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.718876    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.718950    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2xw8t\""
Sep 17 21:44:09.719142 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719139    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 17 21:44:09.719627 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719201    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 17 21:44:09.719627 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719386    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 17 21:44:09.719627 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719465    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-btf6ftqbf1jdi\""
Sep 17 21:44:09.719627 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719603    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 17 21:44:09.719811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.719706    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 17 21:44:09.721144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.721044    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 17 21:44:09.723906 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.723885    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 17 21:44:09.725617 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.725598    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 17 21:44:09.737329 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.737279    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:09.778052 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778016    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778075    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778105    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778129    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778161    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778181    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778205    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778229    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778262    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778302    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778327    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778347    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778382    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778404    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778438    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778469    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778492    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778527    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778554    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.778677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.778583    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfh4d\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-kube-api-access-mfh4d\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879197    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879235    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879258    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfh4d\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-kube-api-access-mfh4d\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879279    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879334    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879352    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879367    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879386    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879402    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879427    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879449    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879486    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879509    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879533    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879557    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879591    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879615    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879651    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879684    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.879958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.879709    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.880422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.880005    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.880422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.880330    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.880422 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.880398    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.881170 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.881128    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.881857 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.881604    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.882003 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.881979    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.882733 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.882708    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.882897 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.882755    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.883482 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.883459    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.883580 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.883531    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884263 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884242    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884359 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884271    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884463    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884631 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884609    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/864a8562-9dea-41b9-ac87-b17483c0dfd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884747 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884728    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.884811 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.884739    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-config\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.885687 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.885671    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.885837 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.885819    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/864a8562-9dea-41b9-ac87-b17483c0dfd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.885950 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.885935    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/864a8562-9dea-41b9-ac87-b17483c0dfd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:09.890565 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:09.890543    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfh4d\" (UniqueName: \"kubernetes.io/projected/864a8562-9dea-41b9-ac87-b17483c0dfd9-kube-api-access-mfh4d\") pod \"prometheus-k8s-0\" (UID: \"864a8562-9dea-41b9-ac87-b17483c0dfd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:10.028062 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.028024    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:44:10.318078 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.318052    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 21:44:10.318796 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:44:10.318772    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864a8562_9dea_41b9_ac87_b17483c0dfd9.slice/crio-f61cdbba599c681b89b667eaaca7e05445eb1c29a0a48b3c959401ae0d2bc2e5 WatchSource:0}: Error finding container f61cdbba599c681b89b667eaaca7e05445eb1c29a0a48b3c959401ae0d2bc2e5: Status 404 returned error can't find the container with id f61cdbba599c681b89b667eaaca7e05445eb1c29a0a48b3c959401ae0d2bc2e5
Sep 17 21:44:10.649651 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.649563    2587 generic.go:356] "Generic (PLEG): container finished" podID="864a8562-9dea-41b9-ac87-b17483c0dfd9" containerID="aabcce067a10d3bf6fabaa49449ddc9bb16010d1d53b3af1e61da9a71d3c6394" exitCode=0
Sep 17 21:44:10.650093 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.649660    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerDied","Data":"aabcce067a10d3bf6fabaa49449ddc9bb16010d1d53b3af1e61da9a71d3c6394"}
Sep 17 21:44:10.650093 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.649749    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"f61cdbba599c681b89b667eaaca7e05445eb1c29a0a48b3c959401ae0d2bc2e5"}
Sep 17 21:44:10.861166 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:10.861133    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a488ee3-f512-4902-b609-5a8b744c5041" path="/var/lib/kubelet/pods/2a488ee3-f512-4902-b609-5a8b744c5041/volumes"
Sep 17 21:44:11.658791 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.658747    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"0f7c4ee63a3ce0aa9e8e2be14830682269c9afc1d0e59631c1b574870f51e34e"}
Sep 17 21:44:11.659154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.658871    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"6d0e4957af6a7d5cfe652749253d6ff8555edcd77314dcffb16aff42305f227e"}
Sep 17 21:44:11.659154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.658910    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"56f8dbffbceb46d5386c45f8df045262c7fbfb155c84e9ff81648debea01bef8"}
Sep 17 21:44:11.659154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.658954    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"0a003740b65e9842acaf399efce62152ebce885ab17e2da0c822b2a0c7330b10"}
Sep 17 21:44:11.659154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.658989    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"f87a756c5202a1f3d39b6a13b104bf12d0395c97a23db68d8fe24081826e34dc"}
Sep 17 21:44:11.659154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.659022    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"864a8562-9dea-41b9-ac87-b17483c0dfd9","Type":"ContainerStarted","Data":"35ec1537d5f9edf297b44842c5ab58034646de6b86d3aa67b35b59ece6680de7"}
Sep 17 21:44:11.695949 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:11.695890    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.666854807 podStartE2EDuration="2.695870608s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.015801ms" isStatefulPod=false podCreationTimestamp="2026-09-17 21:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:44:11.693973422 +0000 UTC m=+155.042707301" watchObservedRunningTime="2026-09-17 21:44:11.695870608 +0000 UTC m=+155.044604489"
Sep 17 21:44:15.029053 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:44:15.029016    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:45:10.029059 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:45:10.029025    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:45:10.045108 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:45:10.045082    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:45:10.909315 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:45:10.909268    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 21:46:27.677416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.677381    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wm7gs"]
Sep 17 21:46:27.679714 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.679696    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.684067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.684041    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 21:46:27.684700 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.684681    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-h6st9\""
Sep 17 21:46:27.684798 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.684686    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 21:46:27.695303 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.695261    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wm7gs"]
Sep 17 21:46:27.756881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.756840    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.757052 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.756906    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ssk\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-kube-api-access-57ssk\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.857772 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.857738    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.857941 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.857835    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ssk\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-kube-api-access-57ssk\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.879217 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.879181    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:27.882386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:27.882360    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ssk\" (UniqueName: \"kubernetes.io/projected/b792aca1-702a-4a85-975d-ece74531ef79-kube-api-access-57ssk\") pod \"cert-manager-webhook-6486ffd997-wm7gs\" (UID: \"b792aca1-702a-4a85-975d-ece74531ef79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:28.008953 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:28.008917    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:28.296724 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:28.296700    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wm7gs"]
Sep 17 21:46:28.298714 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:46:28.298683    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb792aca1_702a_4a85_975d_ece74531ef79.slice/crio-b345f89b465e57cf16c4ac0d9227d812a4b42a817d56e636612225b38dceeace WatchSource:0}: Error finding container b345f89b465e57cf16c4ac0d9227d812a4b42a817d56e636612225b38dceeace: Status 404 returned error can't find the container with id b345f89b465e57cf16c4ac0d9227d812a4b42a817d56e636612225b38dceeace
Sep 17 21:46:29.205027 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:29.204972    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs" event={"ID":"b792aca1-702a-4a85-975d-ece74531ef79","Type":"ContainerStarted","Data":"b345f89b465e57cf16c4ac0d9227d812a4b42a817d56e636612225b38dceeace"}
Sep 17 21:46:31.215526 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:31.215485    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs" event={"ID":"b792aca1-702a-4a85-975d-ece74531ef79","Type":"ContainerStarted","Data":"91a89d56d04fc445f7be36467c6ff6af873a33acbd97fc78a455cc05fdf2bdd8"}
Sep 17 21:46:31.215905 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:31.215621    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:31.238111 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:31.238063    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs" podStartSLOduration=1.495857499 podStartE2EDuration="4.238048509s" totalImagesPullingTime="2.74219101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:46:31.236187083 +0000 UTC m=+294.584920964" watchObservedRunningTime="2026-09-17 21:46:31.238048509 +0000 UTC m=+294.586782388"
Sep 17 21:46:37.221678 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:37.221646    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wm7gs"
Sep 17 21:46:38.729019 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:38.728990    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:46:38.733236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:38.733205    2587 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 21:46:38.735711 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:38.735687    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:46:49.264919 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.264886    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bh6nh"]
Sep 17 21:46:49.268099 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.268083    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.276128 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.276096    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ngf4d\""
Sep 17 21:46:49.283238 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.283214    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bh6nh"]
Sep 17 21:46:49.331672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.331634    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bknmf\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-kube-api-access-bknmf\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.331672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.331673    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-bound-sa-token\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.432964 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.432933    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-bound-sa-token\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.433127 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.433023    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bknmf\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-kube-api-access-bknmf\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.442349 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.442316    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-bound-sa-token\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.442552 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.442530    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bknmf\" (UniqueName: \"kubernetes.io/projected/a0bef6c1-59cc-46a6-9585-fa204cf7c73d-kube-api-access-bknmf\") pod \"cert-manager-55b4d954d5-bh6nh\" (UID: \"a0bef6c1-59cc-46a6-9585-fa204cf7c73d\") " pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.582049 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.581976    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bh6nh"
Sep 17 21:46:49.857960 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:49.857880    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bh6nh"]
Sep 17 21:46:49.860805 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:46:49.860773    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0bef6c1_59cc_46a6_9585_fa204cf7c73d.slice/crio-b36f6a971035f125a20dad18047b190be80037b3c239ee93371ae8b8dc90491e WatchSource:0}: Error finding container b36f6a971035f125a20dad18047b190be80037b3c239ee93371ae8b8dc90491e: Status 404 returned error can't find the container with id b36f6a971035f125a20dad18047b190be80037b3c239ee93371ae8b8dc90491e
Sep 17 21:46:50.299259 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:50.299219    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bh6nh" event={"ID":"a0bef6c1-59cc-46a6-9585-fa204cf7c73d","Type":"ContainerStarted","Data":"55c9af8ddd81d5f4b59c8da49cb0e1494a14495721e080e28650188e6e3bce99"}
Sep 17 21:46:50.299678 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:50.299269    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bh6nh" event={"ID":"a0bef6c1-59cc-46a6-9585-fa204cf7c73d","Type":"ContainerStarted","Data":"b36f6a971035f125a20dad18047b190be80037b3c239ee93371ae8b8dc90491e"}
Sep 17 21:46:50.318718 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:46:50.318662    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-bh6nh" podStartSLOduration=1.318646201 podStartE2EDuration="1.318646201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:46:50.316926518 +0000 UTC m=+313.665660398" watchObservedRunningTime="2026-09-17 21:46:50.318646201 +0000 UTC m=+313.667380081"
Sep 17 21:47:03.438797 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.438764    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"]
Sep 17 21:47:03.442447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.442429    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.447237 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.447214    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 21:47:03.447673 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.447653    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 21:47:03.447777 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.447660    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 21:47:03.447777 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.447721    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 21:47:03.451094 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.451078    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n8qrl\""
Sep 17 21:47:03.465987 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.465967    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"]
Sep 17 21:47:03.548890 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.548859    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-tmp\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.549076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.548900    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-manifests\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.549076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.548951    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.549076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.548971    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.549076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.549045    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb222\" (UniqueName: \"kubernetes.io/projected/325bb420-b689-43ef-ae0a-9aa4565121b7-kube-api-access-hb222\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.650439 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.650408    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.650439 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.650442    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.650650 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.650469    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb222\" (UniqueName: \"kubernetes.io/projected/325bb420-b689-43ef-ae0a-9aa4565121b7-kube-api-access-hb222\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.650650 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.650502    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-tmp\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.650650 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.650520    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-manifests\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.651046 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.651022    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-manifests\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.652805 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.652787    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325bb420-b689-43ef-ae0a-9aa4565121b7-tmp\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.653134 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.653114    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.653175 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.653114    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/325bb420-b689-43ef-ae0a-9aa4565121b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.660634 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.660613    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb222\" (UniqueName: \"kubernetes.io/projected/325bb420-b689-43ef-ae0a-9aa4565121b7-kube-api-access-hb222\") pod \"opendatahub-operator-controller-manager-555cdfd7b4-x9qtq\" (UID: \"325bb420-b689-43ef-ae0a-9aa4565121b7\") " pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:03.752728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:03.752696    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:04.044234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:04.044207    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"]
Sep 17 21:47:04.046213 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:47:04.046182    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325bb420_b689_43ef_ae0a_9aa4565121b7.slice/crio-9475cd39972ec7894ce0416975f5dcf9c4f993c9d2344452ca47fc1967cfae21 WatchSource:0}: Error finding container 9475cd39972ec7894ce0416975f5dcf9c4f993c9d2344452ca47fc1967cfae21: Status 404 returned error can't find the container with id 9475cd39972ec7894ce0416975f5dcf9c4f993c9d2344452ca47fc1967cfae21
Sep 17 21:47:04.048138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:04.048116    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:47:04.358350 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:04.358238    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq" event={"ID":"325bb420-b689-43ef-ae0a-9aa4565121b7","Type":"ContainerStarted","Data":"9475cd39972ec7894ce0416975f5dcf9c4f993c9d2344452ca47fc1967cfae21"}
Sep 17 21:47:07.378124 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:07.378035    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq" event={"ID":"325bb420-b689-43ef-ae0a-9aa4565121b7","Type":"ContainerStarted","Data":"02b72574bfd2b846af14ef4421a5ac07c2e58a4202f1e5ffbc1f3ff42b2d23db"}
Sep 17 21:47:08.384214 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:08.384176    2587 generic.go:356] "Generic (PLEG): container finished" podID="325bb420-b689-43ef-ae0a-9aa4565121b7" containerID="02b72574bfd2b846af14ef4421a5ac07c2e58a4202f1e5ffbc1f3ff42b2d23db" exitCode=0
Sep 17 21:47:08.384591 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:08.384267    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq" event={"ID":"325bb420-b689-43ef-ae0a-9aa4565121b7","Type":"ContainerDied","Data":"02b72574bfd2b846af14ef4421a5ac07c2e58a4202f1e5ffbc1f3ff42b2d23db"}
Sep 17 21:47:09.391674 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:09.391637    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq" event={"ID":"325bb420-b689-43ef-ae0a-9aa4565121b7","Type":"ContainerStarted","Data":"916d4138f9677fe3ba1eb088c2a4301d331a05a9fc62452d01550f95c8831a3d"}
Sep 17 21:47:09.392159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:09.391772    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:09.438672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:09.438613    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq" podStartSLOduration=2.07466138 podStartE2EDuration="6.438594609s" totalImagesPullingTime="3.317795703s" totalInitContainerRuntime="1.046137526s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:47:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:47:09.436365941 +0000 UTC m=+332.785099813" watchObservedRunningTime="2026-09-17 21:47:09.438594609 +0000 UTC m=+332.787328489"
Sep 17 21:47:20.398113 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:20.398034    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555cdfd7b4-x9qtq"
Sep 17 21:47:24.082128 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.082096    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"]
Sep 17 21:47:24.085469 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.085449    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.090183 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.090144    2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"lws-controller-manager-7746dd6866-wjvz6\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" podUID="834453c9-0bae-47c8-b9f8-b306f3d47d64" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.090311 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.090216    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"lws-controller-manager-dockercfg-rczrq\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rczrq\"" type="*v1.Secret"
Sep 17 21:47:24.099863 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.099832    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret"
Sep 17 21:47:24.099863 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.099846    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 17 21:47:24.100007 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.099833    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 17 21:47:24.100007 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.099833    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret"
Sep 17 21:47:24.100007 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:24.099872    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"lws-manager-config\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap"
Sep 17 21:47:24.138228 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.138203    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"]
Sep 17 21:47:24.227900 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.227863    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-metrics-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.228072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.227916    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tz8d\" (UniqueName: \"kubernetes.io/projected/834453c9-0bae-47c8-b9f8-b306f3d47d64-kube-api-access-2tz8d\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.228072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.227952    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.228072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.228000    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/834453c9-0bae-47c8-b9f8-b306f3d47d64-manager-config\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.328883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.328848    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tz8d\" (UniqueName: \"kubernetes.io/projected/834453c9-0bae-47c8-b9f8-b306f3d47d64-kube-api-access-2tz8d\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.328883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.328889    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.329138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.328920    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/834453c9-0bae-47c8-b9f8-b306f3d47d64-manager-config\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.329138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.328954    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-metrics-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:24.768034 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.767998    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"]
Sep 17 21:47:24.771226 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.771207    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:24.778231 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.778212    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 17 21:47:24.778231 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.778225    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 21:47:24.778442 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.778332    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 21:47:24.779825 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.779807    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 17 21:47:24.779902 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.779844    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4k8jx\""
Sep 17 21:47:24.814121 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.814088    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"]
Sep 17 21:47:24.933555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.933526    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5393fa71-d7a8-4c13-855b-f52c839dcee2-tls-certs\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:24.933555 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.933556    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzxz9\" (UniqueName: \"kubernetes.io/projected/5393fa71-d7a8-4c13-855b-f52c839dcee2-kube-api-access-dzxz9\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:24.933848 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.933828    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5393fa71-d7a8-4c13-855b-f52c839dcee2-tmp\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:24.976746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:24.976714    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 21:47:25.034477 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.034376    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5393fa71-d7a8-4c13-855b-f52c839dcee2-tmp\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.034657 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.034493    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5393fa71-d7a8-4c13-855b-f52c839dcee2-tls-certs\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.034657 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.034520    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzxz9\" (UniqueName: \"kubernetes.io/projected/5393fa71-d7a8-4c13-855b-f52c839dcee2-kube-api-access-dzxz9\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.037035 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.037009    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5393fa71-d7a8-4c13-855b-f52c839dcee2-tmp\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.037147 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.037070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5393fa71-d7a8-4c13-855b-f52c839dcee2-tls-certs\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.051385 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.051359    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzxz9\" (UniqueName: \"kubernetes.io/projected/5393fa71-d7a8-4c13-855b-f52c839dcee2-kube-api-access-dzxz9\") pod \"kube-auth-proxy-f7577f47d-r96lt\" (UID: \"5393fa71-d7a8-4c13-855b-f52c839dcee2\") " pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.078543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.078509    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 17 21:47:25.079973 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.079955    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/834453c9-0bae-47c8-b9f8-b306f3d47d64-manager-config\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:25.080678 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.080664    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"
Sep 17 21:47:25.123946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.123911    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rczrq\""
Sep 17 21:47:25.257482 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.257445    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 17 21:47:25.261821 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.261800    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-metrics-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:25.329685 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:25.329606    2587 secret.go:189] Couldn't get secret openshift-lws-operator/webhook-server-cert: failed to sync secret cache: timed out waiting for the condition
Sep 17 21:47:25.329832 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:47:25.329694    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert podName:834453c9-0bae-47c8-b9f8-b306f3d47d64 nodeName:}" failed. No retries permitted until 2026-09-17 21:47:25.829675488 +0000 UTC m=+349.178409346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert") pod "lws-controller-manager-7746dd6866-wjvz6" (UID: "834453c9-0bae-47c8-b9f8-b306f3d47d64") : failed to sync secret cache: timed out waiting for the condition
Sep 17 21:47:25.396732 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.396699    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f7577f47d-r96lt"]
Sep 17 21:47:25.399903 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:47:25.399869    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5393fa71_d7a8_4c13_855b_f52c839dcee2.slice/crio-bcaa4c14c6917cd6a3db50026c0b0ef11aa697f49c0169fd5ebd397f94053762 WatchSource:0}: Error finding container bcaa4c14c6917cd6a3db50026c0b0ef11aa697f49c0169fd5ebd397f94053762: Status 404 returned error can't find the container with id bcaa4c14c6917cd6a3db50026c0b0ef11aa697f49c0169fd5ebd397f94053762
Sep 17 21:47:25.409803 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.409783    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 17 21:47:25.462225 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.462191    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt" event={"ID":"5393fa71-d7a8-4c13-855b-f52c839dcee2","Type":"ContainerStarted","Data":"bcaa4c14c6917cd6a3db50026c0b0ef11aa697f49c0169fd5ebd397f94053762"}
Sep 17 21:47:25.655523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.655438    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 21:47:25.660974 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.660950    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tz8d\" (UniqueName: \"kubernetes.io/projected/834453c9-0bae-47c8-b9f8-b306f3d47d64-kube-api-access-2tz8d\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:25.838494 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.838458    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:25.840998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.840978    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/834453c9-0bae-47c8-b9f8-b306f3d47d64-cert\") pod \"lws-controller-manager-7746dd6866-wjvz6\" (UID: \"834453c9-0bae-47c8-b9f8-b306f3d47d64\") " pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:25.896526 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:25.896499    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:26.190062 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:26.190032    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"]
Sep 17 21:47:26.192867 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:47:26.192835    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834453c9_0bae_47c8_b9f8_b306f3d47d64.slice/crio-316558ee2e03a06d8c6a32cb954bb080ad72088bba586dedc34a70cee6751e6c WatchSource:0}: Error finding container 316558ee2e03a06d8c6a32cb954bb080ad72088bba586dedc34a70cee6751e6c: Status 404 returned error can't find the container with id 316558ee2e03a06d8c6a32cb954bb080ad72088bba586dedc34a70cee6751e6c
Sep 17 21:47:26.469461 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:26.469363    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6" event={"ID":"834453c9-0bae-47c8-b9f8-b306f3d47d64","Type":"ContainerStarted","Data":"316558ee2e03a06d8c6a32cb954bb080ad72088bba586dedc34a70cee6751e6c"}
Sep 17 21:47:30.492855 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:30.492821    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt" event={"ID":"5393fa71-d7a8-4c13-855b-f52c839dcee2","Type":"ContainerStarted","Data":"553ef57975e374811650431dd9fc79520647ea5ad54b346751ee25ae3b1ae05e"}
Sep 17 21:47:30.494509 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:30.494477    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6" event={"ID":"834453c9-0bae-47c8-b9f8-b306f3d47d64","Type":"ContainerStarted","Data":"79bffc9c8bdb310a577131c76c9154db5b90280b9d35a383a3c5c282e2edcdcd"}
Sep 17 21:47:30.494637 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:30.494565    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:47:30.513379 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:30.513338    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f7577f47d-r96lt" podStartSLOduration=2.482002712 podStartE2EDuration="6.513325264s" totalImagesPullingTime="4.031322552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:47:30.51111371 +0000 UTC m=+353.859847593" watchObservedRunningTime="2026-09-17 21:47:30.513325264 +0000 UTC m=+353.862059144"
Sep 17 21:47:30.531070 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:30.531019    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6" podStartSLOduration=3.245389123 podStartE2EDuration="6.531004865s" totalImagesPullingTime="3.285615742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:47:30.529454248 +0000 UTC m=+353.878188142" watchObservedRunningTime="2026-09-17 21:47:30.531004865 +0000 UTC m=+353.879738746"
Sep 17 21:47:41.501666 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:47:41.501634    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7746dd6866-wjvz6"
Sep 17 21:49:13.792869 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.792794    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"]
Sep 17 21:49:13.795003 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.794987    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.798227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.798204    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rlbmk\""
Sep 17 21:49:13.798227 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.798216    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 17 21:49:13.799471 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.799452    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 21:49:13.799471 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.799483    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 21:49:13.799646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.799487    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 17 21:49:13.804769 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.804747    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"]
Sep 17 21:49:13.880098 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.880059    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05748130-db33-4f47-9b2d-3f3e9c28f895-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.880277 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.880115    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05748130-db33-4f47-9b2d-3f3e9c28f895-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.880277 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.880182    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s9xb\" (UniqueName: \"kubernetes.io/projected/05748130-db33-4f47-9b2d-3f3e9c28f895-kube-api-access-5s9xb\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.980920 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.980884    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05748130-db33-4f47-9b2d-3f3e9c28f895-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.981104 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.980935    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05748130-db33-4f47-9b2d-3f3e9c28f895-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.981104 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.980973    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s9xb\" (UniqueName: \"kubernetes.io/projected/05748130-db33-4f47-9b2d-3f3e9c28f895-kube-api-access-5s9xb\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.981553 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.981535    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05748130-db33-4f47-9b2d-3f3e9c28f895-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.983625 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.983606    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05748130-db33-4f47-9b2d-3f3e9c28f895-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:13.991070 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:13.991048    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s9xb\" (UniqueName: \"kubernetes.io/projected/05748130-db33-4f47-9b2d-3f3e9c28f895-kube-api-access-5s9xb\") pod \"kuadrant-console-plugin-fbdd4f445-psg95\" (UID: \"05748130-db33-4f47-9b2d-3f3e9c28f895\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:14.105074 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:14.104981    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"
Sep 17 21:49:14.379359 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:14.379335    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95"]
Sep 17 21:49:14.381187 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:49:14.381158    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05748130_db33_4f47_9b2d_3f3e9c28f895.slice/crio-09107575aaf6c9f0eb5a7292618c2d2d94ab397dcc91d67e92a6119cbbfcb550 WatchSource:0}: Error finding container 09107575aaf6c9f0eb5a7292618c2d2d94ab397dcc91d67e92a6119cbbfcb550: Status 404 returned error can't find the container with id 09107575aaf6c9f0eb5a7292618c2d2d94ab397dcc91d67e92a6119cbbfcb550
Sep 17 21:49:14.979347 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:14.979311    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95" event={"ID":"05748130-db33-4f47-9b2d-3f3e9c28f895","Type":"ContainerStarted","Data":"09107575aaf6c9f0eb5a7292618c2d2d94ab397dcc91d67e92a6119cbbfcb550"}
Sep 17 21:49:19.001724 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:19.001697    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95" event={"ID":"05748130-db33-4f47-9b2d-3f3e9c28f895","Type":"ContainerStarted","Data":"791bc17227c64356633dd25bc7fc82cc58c4e251d1a1ded4aaa4abff98bd2f47"}
Sep 17 21:49:19.032826 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:19.032774    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-psg95" podStartSLOduration=1.486757008 podStartE2EDuration="6.032759132s" totalImagesPullingTime="4.546002124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:49:19.031503843 +0000 UTC m=+462.380237722" watchObservedRunningTime="2026-09-17 21:49:19.032759132 +0000 UTC m=+462.381493012"
Sep 17 21:49:59.425817 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.425780    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:49:59.442482 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.442452    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.446673 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.446643    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:49:59.447953 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.447918    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 17 21:49:59.534929 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.534896    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:49:59.577298 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.577250    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6br5\" (UniqueName: \"kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.577447 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.577349    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.677895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.677806    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6br5\" (UniqueName: \"kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.677895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.677895    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.678526 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.678507    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.695503 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.695474    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6br5\" (UniqueName: \"kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5\") pod \"limitador-limitador-8588749977-ft6sd\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") " pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:49:59.755097 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:49:59.755066    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:50:00.039842 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.039810    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:50:00.042354 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:50:00.042328    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b65dfc6_be25_4047_8412_80a90b3a0548.slice/crio-82af96e13be6ecc964beabd67db8f9a184cfc981117ee5aba7c6121985bfa40d WatchSource:0}: Error finding container 82af96e13be6ecc964beabd67db8f9a184cfc981117ee5aba7c6121985bfa40d: Status 404 returned error can't find the container with id 82af96e13be6ecc964beabd67db8f9a184cfc981117ee5aba7c6121985bfa40d
Sep 17 21:50:00.187869 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.187836    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" event={"ID":"1b65dfc6-be25-4047-8412-80a90b3a0548","Type":"ContainerStarted","Data":"82af96e13be6ecc964beabd67db8f9a184cfc981117ee5aba7c6121985bfa40d"}
Sep 17 21:50:00.625578 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.625541    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:00.630705 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.630680    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:00.633971 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.633946    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bdgv9\""
Sep 17 21:50:00.639231 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.639206    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:00.787548 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.787514    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqld\" (UniqueName: \"kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld\") pod \"authorino-59c974bd48-cfmst\" (UID: \"4ded8e2f-8290-4602-8a1c-a2efa6537fc0\") " pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:00.889004 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.888910    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqld\" (UniqueName: \"kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld\") pod \"authorino-59c974bd48-cfmst\" (UID: \"4ded8e2f-8290-4602-8a1c-a2efa6537fc0\") " pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:00.898597 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.898563    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqld\" (UniqueName: \"kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld\") pod \"authorino-59c974bd48-cfmst\" (UID: \"4ded8e2f-8290-4602-8a1c-a2efa6537fc0\") " pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:00.941437 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:00.941401    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:01.218849 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:01.218380    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:01.222044 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:50:01.222011    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ded8e2f_8290_4602_8a1c_a2efa6537fc0.slice/crio-d65e1cb3f0a27eb14ed318b92b3c52285d33db5e2e55684d1cf137fe2936016f WatchSource:0}: Error finding container d65e1cb3f0a27eb14ed318b92b3c52285d33db5e2e55684d1cf137fe2936016f: Status 404 returned error can't find the container with id d65e1cb3f0a27eb14ed318b92b3c52285d33db5e2e55684d1cf137fe2936016f
Sep 17 21:50:02.201061 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:02.201019    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cfmst" event={"ID":"4ded8e2f-8290-4602-8a1c-a2efa6537fc0","Type":"ContainerStarted","Data":"d65e1cb3f0a27eb14ed318b92b3c52285d33db5e2e55684d1cf137fe2936016f"}
Sep 17 21:50:03.207763 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:03.207718    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" event={"ID":"1b65dfc6-be25-4047-8412-80a90b3a0548","Type":"ContainerStarted","Data":"4c71658f50b8daf7afdb78efb1ef6b2452d10b5bf91e7d151d0e8e2da283caf6"}
Sep 17 21:50:03.208221 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:03.207856    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:50:03.230070 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:03.230010    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" podStartSLOduration=1.6876941859999999 podStartE2EDuration="4.229988326s" totalImagesPullingTime="2.54229414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:50:03.226521104 +0000 UTC m=+506.575254985" watchObservedRunningTime="2026-09-17 21:50:03.229988326 +0000 UTC m=+506.578722209"
Sep 17 21:50:04.216189 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:04.216147    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cfmst" event={"ID":"4ded8e2f-8290-4602-8a1c-a2efa6537fc0","Type":"ContainerStarted","Data":"ad7f402a8842dd1bfde7178accfc8bb639e69874b64e2564ff7a53efbc5c983e"}
Sep 17 21:50:04.236663 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:04.236612    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-cfmst" podStartSLOduration=1.8500748790000001 podStartE2EDuration="4.236595437s" totalImagesPullingTime="2.386520558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:50:04.235753356 +0000 UTC m=+507.584487238" watchObservedRunningTime="2026-09-17 21:50:04.236595437 +0000 UTC m=+507.585329318"
Sep 17 21:50:14.217871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:14.217837    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:50:21.791593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:21.791555    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:50:21.791981 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:21.791759    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" podUID="1b65dfc6-be25-4047-8412-80a90b3a0548" containerName="limitador" containerID="cri-o://4c71658f50b8daf7afdb78efb1ef6b2452d10b5bf91e7d151d0e8e2da283caf6" gracePeriod=30
Sep 17 21:50:22.299509 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.299481    2587 generic.go:356] "Generic (PLEG): container finished" podID="1b65dfc6-be25-4047-8412-80a90b3a0548" containerID="4c71658f50b8daf7afdb78efb1ef6b2452d10b5bf91e7d151d0e8e2da283caf6" exitCode=0
Sep 17 21:50:22.299660 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.299556    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" event={"ID":"1b65dfc6-be25-4047-8412-80a90b3a0548","Type":"ContainerDied","Data":"4c71658f50b8daf7afdb78efb1ef6b2452d10b5bf91e7d151d0e8e2da283caf6"}
Sep 17 21:50:22.337072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.337050    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:50:22.376930 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.376862    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file\") pod \"1b65dfc6-be25-4047-8412-80a90b3a0548\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") "
Sep 17 21:50:22.376930 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.376926    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6br5\" (UniqueName: \"kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5\") pod \"1b65dfc6-be25-4047-8412-80a90b3a0548\" (UID: \"1b65dfc6-be25-4047-8412-80a90b3a0548\") "
Sep 17 21:50:22.377253 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.377194    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file" (OuterVolumeSpecName: "config-file") pod "1b65dfc6-be25-4047-8412-80a90b3a0548" (UID: "1b65dfc6-be25-4047-8412-80a90b3a0548"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:50:22.379233 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.379209    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5" (OuterVolumeSpecName: "kube-api-access-z6br5") pod "1b65dfc6-be25-4047-8412-80a90b3a0548" (UID: "1b65dfc6-be25-4047-8412-80a90b3a0548"). InnerVolumeSpecName "kube-api-access-z6br5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:50:22.478278 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.478243    2587 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b65dfc6-be25-4047-8412-80a90b3a0548-config-file\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:50:22.478278 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:22.478273    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6br5\" (UniqueName: \"kubernetes.io/projected/1b65dfc6-be25-4047-8412-80a90b3a0548-kube-api-access-z6br5\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:50:23.305508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:23.305479    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd"
Sep 17 21:50:23.305508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:23.305504    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ft6sd" event={"ID":"1b65dfc6-be25-4047-8412-80a90b3a0548","Type":"ContainerDied","Data":"82af96e13be6ecc964beabd67db8f9a184cfc981117ee5aba7c6121985bfa40d"}
Sep 17 21:50:23.305940 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:23.305561    2587 scope.go:117] "RemoveContainer" containerID="4c71658f50b8daf7afdb78efb1ef6b2452d10b5bf91e7d151d0e8e2da283caf6"
Sep 17 21:50:23.336981 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:23.336953    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:50:23.354053 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:23.354028    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ft6sd"]
Sep 17 21:50:24.856616 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:24.856586    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b65dfc6-be25-4047-8412-80a90b3a0548" path="/var/lib/kubelet/pods/1b65dfc6-be25-4047-8412-80a90b3a0548/volumes"
Sep 17 21:50:26.163910 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.163874    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"]
Sep 17 21:50:26.164312 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.164274    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b65dfc6-be25-4047-8412-80a90b3a0548" containerName="limitador"
Sep 17 21:50:26.164362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.164314    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b65dfc6-be25-4047-8412-80a90b3a0548" containerName="limitador"
Sep 17 21:50:26.164397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.164372    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b65dfc6-be25-4047-8412-80a90b3a0548" containerName="limitador"
Sep 17 21:50:26.168640 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.168626    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.172881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.172860    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xd7nf\""
Sep 17 21:50:26.173297 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.173267    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 21:50:26.177201 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.177184    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 21:50:26.180038 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.180020    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 17 21:50:26.193680 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.193650    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"]
Sep 17 21:50:26.311812 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.311780    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb686664-145b-4771-936f-3f70fb28884d-data\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.311989 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.311823    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hmfx\" (UniqueName: \"kubernetes.io/projected/fb686664-145b-4771-936f-3f70fb28884d-kube-api-access-4hmfx\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.413073 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.413032    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hmfx\" (UniqueName: \"kubernetes.io/projected/fb686664-145b-4771-936f-3f70fb28884d-kube-api-access-4hmfx\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.413247 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.413125    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb686664-145b-4771-936f-3f70fb28884d-data\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.413509 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.413492    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb686664-145b-4771-936f-3f70fb28884d-data\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.425755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.425703    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hmfx\" (UniqueName: \"kubernetes.io/projected/fb686664-145b-4771-936f-3f70fb28884d-kube-api-access-4hmfx\") pod \"postgres-69fb4594cf-wqm9q\" (UID: \"fb686664-145b-4771-936f-3f70fb28884d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.478735 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.478705    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:26.765993 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:26.765968    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"]
Sep 17 21:50:26.767502 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:50:26.767467    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb686664_145b_4771_936f_3f70fb28884d.slice/crio-8193d26742230839767fcd5f27f1792e6e8b416899d40cf211c2fc496f5545cf WatchSource:0}: Error finding container 8193d26742230839767fcd5f27f1792e6e8b416899d40cf211c2fc496f5545cf: Status 404 returned error can't find the container with id 8193d26742230839767fcd5f27f1792e6e8b416899d40cf211c2fc496f5545cf
Sep 17 21:50:27.326511 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:27.326471    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q" event={"ID":"fb686664-145b-4771-936f-3f70fb28884d","Type":"ContainerStarted","Data":"8193d26742230839767fcd5f27f1792e6e8b416899d40cf211c2fc496f5545cf"}
Sep 17 21:50:32.353934 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:32.353895    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q" event={"ID":"fb686664-145b-4771-936f-3f70fb28884d","Type":"ContainerStarted","Data":"6658173eb33b99542a9bc20229bc98d66d490c4bf3baa227ef8e10e5792fee4a"}
Sep 17 21:50:32.354372 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:32.354020    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:32.386395 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:32.386341    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q" podStartSLOduration=1.681740278 podStartE2EDuration="6.386325673s" totalImagesPullingTime="4.704585395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:50:32.385494164 +0000 UTC m=+535.734228044" watchObservedRunningTime="2026-09-17 21:50:32.386325673 +0000 UTC m=+535.735059552"
Sep 17 21:50:38.387103 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:38.387072    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wqm9q"
Sep 17 21:50:39.120321 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.120230    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-vbns2"]
Sep 17 21:50:39.124925 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.124902    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.128958 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.128938    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 17 21:50:39.137051 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.137032    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-vbns2"]
Sep 17 21:50:39.201105 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.201071    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-vbns2"]
Sep 17 21:50:39.201361 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:50:39.201341    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mb69w tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-vbns2" podUID="530899a4-72ac-4e1b-a520-6cdc8fd47cc8"
Sep 17 21:50:39.237708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.237675    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.237849 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.237765    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb69w\" (UniqueName: \"kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.338196 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.338160    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.338394 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.338244    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb69w\" (UniqueName: \"kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.340895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.340871    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.361006 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.360982    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb69w\" (UniqueName: \"kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w\") pod \"authorino-6d4dfc649f-vbns2\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") " pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.386214 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.386152    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.391989 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.391969    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:39.540057 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.540028    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert\") pod \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") "
Sep 17 21:50:39.540224 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.540066    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb69w\" (UniqueName: \"kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w\") pod \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\" (UID: \"530899a4-72ac-4e1b-a520-6cdc8fd47cc8\") "
Sep 17 21:50:39.542319 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.542274    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w" (OuterVolumeSpecName: "kube-api-access-mb69w") pod "530899a4-72ac-4e1b-a520-6cdc8fd47cc8" (UID: "530899a4-72ac-4e1b-a520-6cdc8fd47cc8"). InnerVolumeSpecName "kube-api-access-mb69w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:50:39.542429 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.542278    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "530899a4-72ac-4e1b-a520-6cdc8fd47cc8" (UID: "530899a4-72ac-4e1b-a520-6cdc8fd47cc8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:50:39.640888 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.640814    2587 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-tls-cert\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:50:39.640888 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:39.640840    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb69w\" (UniqueName: \"kubernetes.io/projected/530899a4-72ac-4e1b-a520-6cdc8fd47cc8-kube-api-access-mb69w\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:50:40.390429 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:40.390398    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-vbns2"
Sep 17 21:50:40.424040 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:40.424014    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-vbns2"]
Sep 17 21:50:40.428595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:40.428575    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-vbns2"]
Sep 17 21:50:40.856771 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:40.856738    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="530899a4-72ac-4e1b-a520-6cdc8fd47cc8" path="/var/lib/kubelet/pods/530899a4-72ac-4e1b-a520-6cdc8fd47cc8/volumes"
Sep 17 21:50:41.248771 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.248731    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:41.248974 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.248950    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-cfmst" podUID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" containerName="authorino" containerID="cri-o://ad7f402a8842dd1bfde7178accfc8bb639e69874b64e2564ff7a53efbc5c983e" gracePeriod=30
Sep 17 21:50:41.397176 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.397143    2587 generic.go:356] "Generic (PLEG): container finished" podID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" containerID="ad7f402a8842dd1bfde7178accfc8bb639e69874b64e2564ff7a53efbc5c983e" exitCode=0
Sep 17 21:50:41.397342 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.397190    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cfmst" event={"ID":"4ded8e2f-8290-4602-8a1c-a2efa6537fc0","Type":"ContainerDied","Data":"ad7f402a8842dd1bfde7178accfc8bb639e69874b64e2564ff7a53efbc5c983e"}
Sep 17 21:50:41.481433 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.481406    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:41.658384 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.658269    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbqld\" (UniqueName: \"kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld\") pod \"4ded8e2f-8290-4602-8a1c-a2efa6537fc0\" (UID: \"4ded8e2f-8290-4602-8a1c-a2efa6537fc0\") "
Sep 17 21:50:41.660538 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.660511    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld" (OuterVolumeSpecName: "kube-api-access-jbqld") pod "4ded8e2f-8290-4602-8a1c-a2efa6537fc0" (UID: "4ded8e2f-8290-4602-8a1c-a2efa6537fc0"). InnerVolumeSpecName "kube-api-access-jbqld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:50:41.759485 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:41.759440    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbqld\" (UniqueName: \"kubernetes.io/projected/4ded8e2f-8290-4602-8a1c-a2efa6537fc0-kube-api-access-jbqld\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:50:42.403175 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.403145    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cfmst"
Sep 17 21:50:42.403175 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.403151    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cfmst" event={"ID":"4ded8e2f-8290-4602-8a1c-a2efa6537fc0","Type":"ContainerDied","Data":"d65e1cb3f0a27eb14ed318b92b3c52285d33db5e2e55684d1cf137fe2936016f"}
Sep 17 21:50:42.403549 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.403196    2587 scope.go:117] "RemoveContainer" containerID="ad7f402a8842dd1bfde7178accfc8bb639e69874b64e2564ff7a53efbc5c983e"
Sep 17 21:50:42.427999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.427975    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:42.431803 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.431780    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cfmst"]
Sep 17 21:50:42.857057 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:42.857019    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" path="/var/lib/kubelet/pods/4ded8e2f-8290-4602-8a1c-a2efa6537fc0/volumes"
Sep 17 21:50:46.175165 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.175134    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:50:46.175718 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.175698    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" containerName="authorino"
Sep 17 21:50:46.175799 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.175722    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" containerName="authorino"
Sep 17 21:50:46.175850 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.175831    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ded8e2f-8290-4602-8a1c-a2efa6537fc0" containerName="authorino"
Sep 17 21:50:46.180084 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.180061    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.183237 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.183215    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 21:50:46.183374 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.183313    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9wpkz\""
Sep 17 21:50:46.183374 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.183335    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 21:50:46.183374 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.183358    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 21:50:46.191245 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.191223    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:50:46.300069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.300038    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stlhj\" (UniqueName: \"kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.300264 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.300075    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.300264 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.300188    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.356113 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.356080    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 21:50:46.359529 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.359508    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.371409 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.371386    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 21:50:46.401536 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.401508    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.401656 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.401565    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stlhj\" (UniqueName: \"kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.401656 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.401588    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.404326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.404305    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.404416 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.404306    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.411646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.411626    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stlhj\" (UniqueName: \"kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj\") pod \"maas-controller-7d5f786cd4-7svdx\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") " pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.491888 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.491863    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:46.502019 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.501995    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw42h\" (UniqueName: \"kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.502192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.502028    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.502192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.502048    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.602679 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.602652    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw42h\" (UniqueName: \"kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.602856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.602687    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.602856 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.602714    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.605248 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.605222    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.605390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.605322    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.612750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.612725    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw42h\" (UniqueName: \"kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h\") pod \"maas-controller-b978497bd-l7kd7\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") " pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.669635 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.669607    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:46.763016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.762992    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:50:46.764901 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:50:46.764872    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a6cd8b_f618_4850_8dbf_bf8e5e75f3fd.slice/crio-b7a1ed4aaf5bf19cd056435003717026c920ce59f3d21bafe8570e76420ee941 WatchSource:0}: Error finding container b7a1ed4aaf5bf19cd056435003717026c920ce59f3d21bafe8570e76420ee941: Status 404 returned error can't find the container with id b7a1ed4aaf5bf19cd056435003717026c920ce59f3d21bafe8570e76420ee941
Sep 17 21:50:46.942902 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:46.942874    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 21:50:46.945255 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:50:46.945224    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13488348_10bf_41c5_8aed_102668246b5d.slice/crio-01a7e9511afef820361517ad4cff97484deb78db8238f4934d91e127d6fa1707 WatchSource:0}: Error finding container 01a7e9511afef820361517ad4cff97484deb78db8238f4934d91e127d6fa1707: Status 404 returned error can't find the container with id 01a7e9511afef820361517ad4cff97484deb78db8238f4934d91e127d6fa1707
Sep 17 21:50:47.428580 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:47.428541    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-l7kd7" event={"ID":"13488348-10bf-41c5-8aed-102668246b5d","Type":"ContainerStarted","Data":"01a7e9511afef820361517ad4cff97484deb78db8238f4934d91e127d6fa1707"}
Sep 17 21:50:47.429744 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:47.429722    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" event={"ID":"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd","Type":"ContainerStarted","Data":"b7a1ed4aaf5bf19cd056435003717026c920ce59f3d21bafe8570e76420ee941"}
Sep 17 21:50:50.447883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.447841    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" event={"ID":"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd","Type":"ContainerStarted","Data":"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"}
Sep 17 21:50:50.448392 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.447927    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:50:50.449409 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.449387    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-l7kd7" event={"ID":"13488348-10bf-41c5-8aed-102668246b5d","Type":"ContainerStarted","Data":"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"}
Sep 17 21:50:50.449518 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.449509    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:50:50.496168 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.496115    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" podStartSLOduration=1.562189306 podStartE2EDuration="4.496098875s" totalImagesPullingTime="2.933909569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:50:50.495058528 +0000 UTC m=+553.843792410" watchObservedRunningTime="2026-09-17 21:50:50.496098875 +0000 UTC m=+553.844832758"
Sep 17 21:50:50.522931 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:50:50.522877    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b978497bd-l7kd7" podStartSLOduration=1.778505747 podStartE2EDuration="4.522861701s" totalImagesPullingTime="2.744355954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:50:50.520449952 +0000 UTC m=+553.869183831" watchObservedRunningTime="2026-09-17 21:50:50.522861701 +0000 UTC m=+553.871595580"
Sep 17 21:51:01.456335 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.456306    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:51:01.456711 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.456375    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 21:51:01.553637 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.553602    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:51:01.553804 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.553788    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" podUID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" containerName="manager" containerID="cri-o://c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329" gracePeriod=30
Sep 17 21:51:01.802448 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.802418    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:51:01.936563 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.936526    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert\") pod \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") "
Sep 17 21:51:01.936761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.936596    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stlhj\" (UniqueName: \"kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj\") pod \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") "
Sep 17 21:51:01.936761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.936627    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs\") pod \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\" (UID: \"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd\") "
Sep 17 21:51:01.938982 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.938954    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" (UID: "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:51:01.939076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.938994    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj" (OuterVolumeSpecName: "kube-api-access-stlhj") pod "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" (UID: "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd"). InnerVolumeSpecName "kube-api-access-stlhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:51:01.939076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:01.939019    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" (UID: "15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:51:02.037854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.037769    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stlhj\" (UniqueName: \"kubernetes.io/projected/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-kube-api-access-stlhj\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:51:02.037854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.037799    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:51:02.037854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.037809    2587 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd-webhook-cert\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:51:02.505919 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.505880    2587 generic.go:356] "Generic (PLEG): container finished" podID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" containerID="c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329" exitCode=0
Sep 17 21:51:02.505919 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.505919    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" event={"ID":"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd","Type":"ContainerDied","Data":"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"}
Sep 17 21:51:02.506390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.505932    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5f786cd4-7svdx"
Sep 17 21:51:02.506390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.505968    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5f786cd4-7svdx" event={"ID":"15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd","Type":"ContainerDied","Data":"b7a1ed4aaf5bf19cd056435003717026c920ce59f3d21bafe8570e76420ee941"}
Sep 17 21:51:02.506390 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.505994    2587 scope.go:117] "RemoveContainer" containerID="c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"
Sep 17 21:51:02.514744 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.514725    2587 scope.go:117] "RemoveContainer" containerID="c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"
Sep 17 21:51:02.515009 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:51:02.514989    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329\": container with ID starting with c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329 not found: ID does not exist" containerID="c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"
Sep 17 21:51:02.515056 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.515018    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329"} err="failed to get container status \"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329\": rpc error: code = NotFound desc = could not find container \"c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329\": container with ID starting with c420c8ff46898c5846f2e40b88f1ad1c3685bcbfce508b87cd296f810bcdd329 not found: ID does not exist"
Sep 17 21:51:02.533592 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.531242    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:51:02.536472 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.536448    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d5f786cd4-7svdx"]
Sep 17 21:51:02.857757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:02.857667    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" path="/var/lib/kubelet/pods/15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd/volumes"
Sep 17 21:51:26.690699 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.690661    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 21:51:26.691275 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.691239    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" containerName="manager"
Sep 17 21:51:26.691275 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.691259    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" containerName="manager"
Sep 17 21:51:26.691441 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.691364    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a6cd8b-f618-4850-8dbf-bf8e5e75f3fd" containerName="manager"
Sep 17 21:51:26.695975 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.695957    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.699022 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.699001    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 21:51:26.699140 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.699123    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 17 21:51:26.699984 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.699947    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bvxx5\""
Sep 17 21:51:26.710891 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.710868    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 21:51:26.768356 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.768321    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.768498 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.768385    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf8sg\" (UniqueName: \"kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.768498 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.768414    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.768569 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.768494    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.869942 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.869907    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.870096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.869960    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.870096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.870002    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf8sg\" (UniqueName: \"kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.870096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.870026    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.870196 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:51:26.870126    2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 17 21:51:26.870196 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:51:26.870190    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls podName:3b29350a-c3ee-464e-aa6f-c250c68bd756 nodeName:}" failed. No retries permitted until 2026-09-17 21:51:27.370171393 +0000 UTC m=+590.718905251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls") pod "maas-api-d48fb5469-zq6hw" (UID: "3b29350a-c3ee-464e-aa6f-c250c68bd756") : secret "maas-api-serving-cert" not found
Sep 17 21:51:26.870751 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.870733    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.872696 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.872675    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:26.880204 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:26.880182    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf8sg\" (UniqueName: \"kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:27.373865 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.373825    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:27.376532 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.376504    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") pod \"maas-api-d48fb5469-zq6hw\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") " pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:27.489604 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.489571    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"]
Sep 17 21:51:27.493069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.493052    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.507754 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.507734    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-88ctk\""
Sep 17 21:51:27.525588 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.525564    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 17 21:51:27.550178 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.550146    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"]
Sep 17 21:51:27.575976 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.575945    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kc67\" (UniqueName: \"kubernetes.io/projected/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-kube-api-access-7kc67\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.575976 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.575975    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.608690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.608658    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:27.676780 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.676458    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kc67\" (UniqueName: \"kubernetes.io/projected/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-kube-api-access-7kc67\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.676780 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.676494    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.677114 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.677096    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.700644 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.700616    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kc67\" (UniqueName: \"kubernetes.io/projected/a0f27f4a-5e01-4eac-8cad-f99a3908ded8-kube-api-access-7kc67\") pod \"payload-processing-5b54b7fbb7-hzhpk\" (UID: \"a0f27f4a-5e01-4eac-8cad-f99a3908ded8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.802753 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.802713    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:27.917765 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:27.917735    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 21:51:27.919912 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:51:27.919879    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b29350a_c3ee_464e_aa6f_c250c68bd756.slice/crio-ebcf97bc6d5e4b4f7acaad775770a9b59c21ee89f010680ce8d9f866e6e5e2bd WatchSource:0}: Error finding container ebcf97bc6d5e4b4f7acaad775770a9b59c21ee89f010680ce8d9f866e6e5e2bd: Status 404 returned error can't find the container with id ebcf97bc6d5e4b4f7acaad775770a9b59c21ee89f010680ce8d9f866e6e5e2bd
Sep 17 21:51:28.131638 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:28.131616    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"]
Sep 17 21:51:28.133597 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:51:28.133569    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f27f4a_5e01_4eac_8cad_f99a3908ded8.slice/crio-44458f0c8b8f3f38d0267847a4ebdf409e79a54ce076373f8720ec1350c307a0 WatchSource:0}: Error finding container 44458f0c8b8f3f38d0267847a4ebdf409e79a54ce076373f8720ec1350c307a0: Status 404 returned error can't find the container with id 44458f0c8b8f3f38d0267847a4ebdf409e79a54ce076373f8720ec1350c307a0
Sep 17 21:51:28.635689 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:28.635651    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" event={"ID":"3b29350a-c3ee-464e-aa6f-c250c68bd756","Type":"ContainerStarted","Data":"ebcf97bc6d5e4b4f7acaad775770a9b59c21ee89f010680ce8d9f866e6e5e2bd"}
Sep 17 21:51:28.636946 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:28.636920    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk" event={"ID":"a0f27f4a-5e01-4eac-8cad-f99a3908ded8","Type":"ContainerStarted","Data":"44458f0c8b8f3f38d0267847a4ebdf409e79a54ce076373f8720ec1350c307a0"}
Sep 17 21:51:30.647796 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.647740    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" event={"ID":"3b29350a-c3ee-464e-aa6f-c250c68bd756","Type":"ContainerStarted","Data":"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"}
Sep 17 21:51:30.648330 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.647850    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:30.649411 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.649387    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk" event={"ID":"a0f27f4a-5e01-4eac-8cad-f99a3908ded8","Type":"ContainerStarted","Data":"c12d0070ea8b078e616907563a4b0b740457b9fef22b1ef65d78d153fadd0385"}
Sep 17 21:51:30.649525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.649514    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:51:30.668999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.668953    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" podStartSLOduration=2.777491017 podStartE2EDuration="4.668939487s" totalImagesPullingTime="1.89144847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:51:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:51:30.667912253 +0000 UTC m=+594.016646138" watchObservedRunningTime="2026-09-17 21:51:30.668939487 +0000 UTC m=+594.017673680"
Sep 17 21:51:30.694843 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:30.694795    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk" podStartSLOduration=2.015514453 podStartE2EDuration="3.694782692s" totalImagesPullingTime="1.679268239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:51:30.694214775 +0000 UTC m=+594.042948656" watchObservedRunningTime="2026-09-17 21:51:30.694782692 +0000 UTC m=+594.043516572"
Sep 17 21:51:36.660067 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:36.660036    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 21:51:38.764048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:38.764022    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:51:38.768404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:38.768384    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:51:41.656890 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:51:41.656855    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hzhpk"
Sep 17 21:52:19.410008 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.409974    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"]
Sep 17 21:52:19.413363 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.413334    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.419260 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.419234    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v95hl\""
Sep 17 21:52:19.419404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.419271    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 21:52:19.419404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.419270    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 17 21:52:19.419535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.419436    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 21:52:19.439031 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.439003    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"]
Sep 17 21:52:19.442362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442336    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq862\" (UniqueName: \"kubernetes.io/projected/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-kube-api-access-nq862\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.442477 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442421    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.442477 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442443    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.442582 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442482    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.442582 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442529    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.442582 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.442569    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.543909 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.543873    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544088 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.543914    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544088 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.543962    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544088 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.543985    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544088 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.544021    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544088 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.544066    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq862\" (UniqueName: \"kubernetes.io/projected/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-kube-api-access-nq862\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544434 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.544409    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.544461    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.544519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.544491    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.546404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.546378    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.546525 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.546509    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.556199 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.556176    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq862\" (UniqueName: \"kubernetes.io/projected/26786f2f-3a0b-4bfe-bfb2-d00dc915f968-kube-api-access-nq862\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mb2fr\" (UID: \"26786f2f-3a0b-4bfe-bfb2-d00dc915f968\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:19.723848 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:19.723807    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:20.044351 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:20.044319    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"]
Sep 17 21:52:20.045986 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:52:20.045952    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26786f2f_3a0b_4bfe_bfb2_d00dc915f968.slice/crio-2ec3c7660ab3cca880f39f1c0f1772b3372cf0f9c343b8d074c6ca3d58d45163 WatchSource:0}: Error finding container 2ec3c7660ab3cca880f39f1c0f1772b3372cf0f9c343b8d074c6ca3d58d45163: Status 404 returned error can't find the container with id 2ec3c7660ab3cca880f39f1c0f1772b3372cf0f9c343b8d074c6ca3d58d45163
Sep 17 21:52:20.047373 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:20.047353    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:52:20.885138 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:20.885098    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr" event={"ID":"26786f2f-3a0b-4bfe-bfb2-d00dc915f968","Type":"ContainerStarted","Data":"2ec3c7660ab3cca880f39f1c0f1772b3372cf0f9c343b8d074c6ca3d58d45163"}
Sep 17 21:52:21.892538 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:21.892501    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr" event={"ID":"26786f2f-3a0b-4bfe-bfb2-d00dc915f968","Type":"ContainerStarted","Data":"b3685d2744b7f9ab549b0ce17f4681bba1f5dbb8411d87ac64100e48f60312f6"}
Sep 17 21:52:21.918049 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:21.917998    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr" podStartSLOduration=1.911131138 podStartE2EDuration="2.917979231s" totalImagesPullingTime="1.006848093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:52:21.915205684 +0000 UTC m=+645.263939570" watchObservedRunningTime="2026-09-17 21:52:21.917979231 +0000 UTC m=+645.266713117"
Sep 17 21:52:29.724899 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:29.724862    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:29.725305 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:29.724921    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:29.737345 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:29.737320    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:52:29.941749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:52:29.941713    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mb2fr"
Sep 17 21:53:07.750784 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.750750    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"]
Sep 17 21:53:07.753823 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.753806    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.758399 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.758372    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 17 21:53:07.761816 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.761431    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"]
Sep 17 21:53:07.783049 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783017    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b74b9b7-41e5-488b-84c4-fda917e61859-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.783172 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783103    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.783172 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783137    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7hdq\" (UniqueName: \"kubernetes.io/projected/0b74b9b7-41e5-488b-84c4-fda917e61859-kube-api-access-m7hdq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.783316 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783190    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.783316 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783274    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.783425 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.783342    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.884799 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.884764    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b74b9b7-41e5-488b-84c4-fda917e61859-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.884953 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.884820    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.884953 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.884930    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7hdq\" (UniqueName: \"kubernetes.io/projected/0b74b9b7-41e5-488b-84c4-fda917e61859-kube-api-access-m7hdq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885085 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.884965    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885085 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.884991    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885085 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.885024    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885239 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.885198    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885352 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.885270    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.885394 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.885377    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.887262 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.887239    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b74b9b7-41e5-488b-84c4-fda917e61859-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.887643 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.887621    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b74b9b7-41e5-488b-84c4-fda917e61859-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:07.895183 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:07.895158    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7hdq\" (UniqueName: \"kubernetes.io/projected/0b74b9b7-41e5-488b-84c4-fda917e61859-kube-api-access-m7hdq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd\" (UID: \"0b74b9b7-41e5-488b-84c4-fda917e61859\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:08.066864 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:08.066776    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:08.345517 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:08.345492    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"]
Sep 17 21:53:08.347253 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:53:08.347221    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b74b9b7_41e5_488b_84c4_fda917e61859.slice/crio-10d5421bf4d3c0bf3c7f7c35b06bb1cd79fe291338563146db9fe816552cfe42 WatchSource:0}: Error finding container 10d5421bf4d3c0bf3c7f7c35b06bb1cd79fe291338563146db9fe816552cfe42: Status 404 returned error can't find the container with id 10d5421bf4d3c0bf3c7f7c35b06bb1cd79fe291338563146db9fe816552cfe42
Sep 17 21:53:09.120162 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:09.120122    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd" event={"ID":"0b74b9b7-41e5-488b-84c4-fda917e61859","Type":"ContainerStarted","Data":"73b2d1c438873c310a60cfccb78cccceab2807bcd62fd42a871618e3615e3651"}
Sep 17 21:53:09.120757 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:09.120188    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd" event={"ID":"0b74b9b7-41e5-488b-84c4-fda917e61859","Type":"ContainerStarted","Data":"10d5421bf4d3c0bf3c7f7c35b06bb1cd79fe291338563146db9fe816552cfe42"}
Sep 17 21:53:09.139972 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:09.139925    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd" podStartSLOduration=1.949638271 podStartE2EDuration="2.139911363s" totalImagesPullingTime="190.273092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:53:09.138155671 +0000 UTC m=+692.486889553" watchObservedRunningTime="2026-09-17 21:53:09.139911363 +0000 UTC m=+692.488645242"
Sep 17 21:53:18.067065 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:18.067028    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:18.067487 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:18.067091    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:18.079478 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:18.079455    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:18.172031 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:18.172001    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd"
Sep 17 21:53:52.076212 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.076178    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"]
Sep 17 21:53:52.079672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.079656    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.083551 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.083528    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 17 21:53:52.106003 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.105978    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"]
Sep 17 21:53:52.196932 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.196905    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.197115 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.196934    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.197115 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.196975    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.197115 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.196992    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtxm4\" (UniqueName: \"kubernetes.io/projected/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-kube-api-access-vtxm4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.197115 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.197075    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.197365 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.197165    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298025 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.297992    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298186 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298031    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298186 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298074    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298186 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298098    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtxm4\" (UniqueName: \"kubernetes.io/projected/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-kube-api-access-vtxm4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298186 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298144    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298414 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298208    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298454 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298438    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298517 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298494    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.298617 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.298586    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.300580 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.300558    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.300749 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.300730    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.308271 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.308248    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtxm4\" (UniqueName: \"kubernetes.io/projected/69c2dd6a-7aee-43d6-b2f1-4896088dd92a-kube-api-access-vtxm4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4\" (UID: \"69c2dd6a-7aee-43d6-b2f1-4896088dd92a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.389440 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.389347    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:53:52.664944 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:52.664920    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"]
Sep 17 21:53:52.667337 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:53:52.667311    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69c2dd6a_7aee_43d6_b2f1_4896088dd92a.slice/crio-15d32448d0074b2393d6e6b5b00cd7512b91339e9cd7dfda018210e89ec558a8 WatchSource:0}: Error finding container 15d32448d0074b2393d6e6b5b00cd7512b91339e9cd7dfda018210e89ec558a8: Status 404 returned error can't find the container with id 15d32448d0074b2393d6e6b5b00cd7512b91339e9cd7dfda018210e89ec558a8
Sep 17 21:53:53.335818 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:53.335780    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4" event={"ID":"69c2dd6a-7aee-43d6-b2f1-4896088dd92a","Type":"ContainerStarted","Data":"c7d4e021293be94d2a7ec9a7f1446847540edf5ccf5a5afbc4d613b100b63580"}
Sep 17 21:53:53.336185 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:53.335826    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4" event={"ID":"69c2dd6a-7aee-43d6-b2f1-4896088dd92a","Type":"ContainerStarted","Data":"15d32448d0074b2393d6e6b5b00cd7512b91339e9cd7dfda018210e89ec558a8"}
Sep 17 21:53:53.355668 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:53:53.355623    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4" podStartSLOduration=1.180441348 podStartE2EDuration="1.355605487s" totalImagesPullingTime="175.164139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:53:53.355012162 +0000 UTC m=+736.703746046" watchObservedRunningTime="2026-09-17 21:53:53.355605487 +0000 UTC m=+736.704339370"
Sep 17 21:54:02.389725 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:54:02.389690    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:54:02.390175 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:54:02.389742    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:54:02.402042 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:54:02.402018    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:54:03.398303 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:54:03.398267    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4"
Sep 17 21:55:11.904513 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:11.904439    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fd68bd744-hc96m"]
Sep 17 21:55:11.907870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:11.907854    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:11.911969 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:11.911949    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 17 21:55:11.912095 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:11.911993    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bdgv9\""
Sep 17 21:55:11.917397 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:11.917379    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd68bd744-hc96m"]
Sep 17 21:55:12.028965 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.028926    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7efd9b25-1440-45ed-b0f6-57d969bf6351-tls-cert\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.028965 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.028966    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bqmn\" (UniqueName: \"kubernetes.io/projected/7efd9b25-1440-45ed-b0f6-57d969bf6351-kube-api-access-2bqmn\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.130219 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.130188    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7efd9b25-1440-45ed-b0f6-57d969bf6351-tls-cert\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.130219 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.130222    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bqmn\" (UniqueName: \"kubernetes.io/projected/7efd9b25-1440-45ed-b0f6-57d969bf6351-kube-api-access-2bqmn\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.132978 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.132948    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7efd9b25-1440-45ed-b0f6-57d969bf6351-tls-cert\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.142122 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.142099    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bqmn\" (UniqueName: \"kubernetes.io/projected/7efd9b25-1440-45ed-b0f6-57d969bf6351-kube-api-access-2bqmn\") pod \"authorino-fd68bd744-hc96m\" (UID: \"7efd9b25-1440-45ed-b0f6-57d969bf6351\") " pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.217881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.217828    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd68bd744-hc96m"
Sep 17 21:55:12.508490 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.508467    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd68bd744-hc96m"]
Sep 17 21:55:12.511088 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:12.511061    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7efd9b25_1440_45ed_b0f6_57d969bf6351.slice/crio-79f7a93dcfdb676400d2fd07a25238dab9bf021746b15cdbad9beb03f836b245 WatchSource:0}: Error finding container 79f7a93dcfdb676400d2fd07a25238dab9bf021746b15cdbad9beb03f836b245: Status 404 returned error can't find the container with id 79f7a93dcfdb676400d2fd07a25238dab9bf021746b15cdbad9beb03f836b245
Sep 17 21:55:12.729021 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:12.728987    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd68bd744-hc96m" event={"ID":"7efd9b25-1440-45ed-b0f6-57d969bf6351","Type":"ContainerStarted","Data":"79f7a93dcfdb676400d2fd07a25238dab9bf021746b15cdbad9beb03f836b245"}
Sep 17 21:55:13.736722 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:13.736682    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd68bd744-hc96m" event={"ID":"7efd9b25-1440-45ed-b0f6-57d969bf6351","Type":"ContainerStarted","Data":"86b637347cafedb7f0b6eca31a2244c7886d18e624886b40e51908aba9638857"}
Sep 17 21:55:13.779192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:13.779134    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fd68bd744-hc96m" podStartSLOduration=2.22334622 podStartE2EDuration="2.779117161s" totalImagesPullingTime="555.770941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:13.776403988 +0000 UTC m=+817.125137870" watchObservedRunningTime="2026-09-17 21:55:13.779117161 +0000 UTC m=+817.127851041"
Sep 17 21:55:36.933439 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:36.933408    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 21:55:36.937034 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:36.937015    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:36.940537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:36.940513    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-f996ae-serving-cert\""
Sep 17 21:55:36.959302 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:36.959260    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 21:55:37.066738 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.066703    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.066936 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.066769    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.066936 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.066801    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9lg\" (UniqueName: \"kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.066936 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.066843    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.167883 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.167847    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.168073 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.167921    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.168073 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.167966    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.168073 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.167983    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9lg\" (UniqueName: \"kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.168244 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:55:37.168089    2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-serving-cert: secret "maas-api-e2e-worker-w0-f996ae-serving-cert" not found
Sep 17 21:55:37.168244 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:55:37.168166    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls podName:583517c8-0e05-4e10-bb7f-886806dfaab2 nodeName:}" failed. No retries permitted until 2026-09-17 21:55:37.668147078 +0000 UTC m=+841.016880940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls") pod "maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" (UID: "583517c8-0e05-4e10-bb7f-886806dfaab2") : secret "maas-api-e2e-worker-w0-f996ae-serving-cert" not found
Sep 17 21:55:37.168695 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.168671    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.170535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.170517    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.216031 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.215962    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9lg\" (UniqueName: \"kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.674007 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.673958    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.676612 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.676591    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:37.848305 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:37.848247    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:38.131886 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:38.131852    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583517c8_0e05_4e10_bb7f_886806dfaab2.slice/crio-34ef0e925fe274014b5c6947fd298c2a59e2052ed4b323f19b78325babba0223 WatchSource:0}: Error finding container 34ef0e925fe274014b5c6947fd298c2a59e2052ed4b323f19b78325babba0223: Status 404 returned error can't find the container with id 34ef0e925fe274014b5c6947fd298c2a59e2052ed4b323f19b78325babba0223
Sep 17 21:55:38.140440 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.140413    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 21:55:38.465210 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.465111    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 21:55:38.470046 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.470023    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.481064 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.481040    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-f996ae-dockercfg-ssnbg\""
Sep 17 21:55:38.481256 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.481238    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-f996ae\""
Sep 17 21:55:38.511214 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.511182    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 21:55:38.582024 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.581991    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf7cn\" (UniqueName: \"kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.582148 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.582130    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.683412 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.683374    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf7cn\" (UniqueName: \"kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.683572 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.683439    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.687484 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.687467    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-f996ae\""
Sep 17 21:55:38.694278 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.694252    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.699005 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.698977    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf7cn\" (UniqueName: \"kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn\") pod \"payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.783773 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.783736    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-f996ae-dockercfg-ssnbg\""
Sep 17 21:55:38.791649 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.791621    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:38.862240 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.862203    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" event={"ID":"583517c8-0e05-4e10-bb7f-886806dfaab2","Type":"ContainerStarted","Data":"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"}
Sep 17 21:55:38.862396 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.862277    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:38.862396 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.862323    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" event={"ID":"583517c8-0e05-4e10-bb7f-886806dfaab2","Type":"ContainerStarted","Data":"34ef0e925fe274014b5c6947fd298c2a59e2052ed4b323f19b78325babba0223"}
Sep 17 21:55:38.909698 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:38.909638    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" podStartSLOduration=2.484495098 podStartE2EDuration="2.909622732s" totalImagesPullingTime="425.127634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:38.908352946 +0000 UTC m=+842.257086827" watchObservedRunningTime="2026-09-17 21:55:38.909622732 +0000 UTC m=+842.258356611"
Sep 17 21:55:39.083610 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:39.083584    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 21:55:39.085200 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:39.085169    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443259d0_308d_4f82_9d61_6ae6db6bb569.slice/crio-1863202c9c7e3de16548648c5577f644af2380a0bdf52839197e2d6de4a37f0d WatchSource:0}: Error finding container 1863202c9c7e3de16548648c5577f644af2380a0bdf52839197e2d6de4a37f0d: Status 404 returned error can't find the container with id 1863202c9c7e3de16548648c5577f644af2380a0bdf52839197e2d6de4a37f0d
Sep 17 21:55:39.867672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:39.867630    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" event={"ID":"443259d0-308d-4f82-9d61-6ae6db6bb569","Type":"ContainerStarted","Data":"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"}
Sep 17 21:55:39.868159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:39.867695    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" event={"ID":"443259d0-308d-4f82-9d61-6ae6db6bb569","Type":"ContainerStarted","Data":"1863202c9c7e3de16548648c5577f644af2380a0bdf52839197e2d6de4a37f0d"}
Sep 17 21:55:39.868159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:39.867750    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:39.961844 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:39.961789    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" podStartSLOduration=1.961771903 podStartE2EDuration="1.961771903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:39.957608476 +0000 UTC m=+843.306342391" watchObservedRunningTime="2026-09-17 21:55:39.961771903 +0000 UTC m=+843.310505782"
Sep 17 21:55:42.485901 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.485860    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 21:55:42.490537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.490518    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.503273 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.503252    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-dbe8a7-serving-cert\""
Sep 17 21:55:42.518011 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.517981    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.518154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.518032    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6tqg\" (UniqueName: \"kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.518154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.518083    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.518154 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.518127    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.526231 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.526207    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 21:55:42.619720 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.619676    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.619895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.619742    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6tqg\" (UniqueName: \"kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.619895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.619778    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.619895 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.619812    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.619895 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:55:42.619843    2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-serving-cert: secret "maas-api-e2e-worker-w4-dbe8a7-serving-cert" not found
Sep 17 21:55:42.620111 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:55:42.619920    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls podName:7c4b72a4-7945-4ce7-b62b-6f9fd89babe8 nodeName:}" failed. No retries permitted until 2026-09-17 21:55:43.119901235 +0000 UTC m=+846.468635109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls") pod "maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" (UID: "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8") : secret "maas-api-e2e-worker-w4-dbe8a7-serving-cert" not found
Sep 17 21:55:42.620538 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.620515    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.622378 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.622351    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:42.637678 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:42.637650    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6tqg\" (UniqueName: \"kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:43.125102 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:43.125061    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:43.127677 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:43.127645    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:43.402035 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:43.401941    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:43.744563 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:43.744528    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 21:55:43.753535 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:43.753491    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4b72a4_7945_4ce7_b62b_6f9fd89babe8.slice/crio-fa8245bf70cfe374c7101e1e4a34fb67a276b6d23f91f2195d757fea1c0a4f85 WatchSource:0}: Error finding container fa8245bf70cfe374c7101e1e4a34fb67a276b6d23f91f2195d757fea1c0a4f85: Status 404 returned error can't find the container with id fa8245bf70cfe374c7101e1e4a34fb67a276b6d23f91f2195d757fea1c0a4f85
Sep 17 21:55:43.890881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:43.890845    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" event={"ID":"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8","Type":"ContainerStarted","Data":"fa8245bf70cfe374c7101e1e4a34fb67a276b6d23f91f2195d757fea1c0a4f85"}
Sep 17 21:55:44.325423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.325389    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 21:55:44.328910 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.328891    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.334491 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.334465    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-dbe8a7\""
Sep 17 21:55:44.366720 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.366633    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 21:55:44.439252 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.439222    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.439491 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.439342    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwt58\" (UniqueName: \"kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.540455 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.540418    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.540632 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.540507    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwt58\" (UniqueName: \"kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.541408 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.541388    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.569539 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.569503    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwt58\" (UniqueName: \"kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58\") pod \"payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.639198 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.639116    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:44.899419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.899326    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" event={"ID":"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8","Type":"ContainerStarted","Data":"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"}
Sep 17 21:55:44.899810 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.899498    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:44.948096 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.948067    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 21:55:44.953070 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:44.953038    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81361305_1d90_4ab7_a7a4_3f1f138436c8.slice/crio-1fd21b4300d4855d12422071ea925ab3eba746427238228e2e5fc83bd8b27761 WatchSource:0}: Error finding container 1fd21b4300d4855d12422071ea925ab3eba746427238228e2e5fc83bd8b27761: Status 404 returned error can't find the container with id 1fd21b4300d4855d12422071ea925ab3eba746427238228e2e5fc83bd8b27761
Sep 17 21:55:44.963631 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:44.963588    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" podStartSLOduration=2.613556642 podStartE2EDuration="2.963572075s" totalImagesPullingTime="350.015433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:44.962057038 +0000 UTC m=+848.310790922" watchObservedRunningTime="2026-09-17 21:55:44.963572075 +0000 UTC m=+848.312305956"
Sep 17 21:55:45.879987 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:45.879954    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 21:55:45.909651 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:45.909610    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" event={"ID":"81361305-1d90-4ab7-a7a4-3f1f138436c8","Type":"ContainerStarted","Data":"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"}
Sep 17 21:55:45.910104 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:45.909682    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" event={"ID":"81361305-1d90-4ab7-a7a4-3f1f138436c8","Type":"ContainerStarted","Data":"1fd21b4300d4855d12422071ea925ab3eba746427238228e2e5fc83bd8b27761"}
Sep 17 21:55:45.910104 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:45.909963    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:45.996634 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:45.996561    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" podStartSLOduration=1.996540054 podStartE2EDuration="1.996540054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:45.990570878 +0000 UTC m=+849.339304762" watchObservedRunningTime="2026-09-17 21:55:45.996540054 +0000 UTC m=+849.345273935"
Sep 17 21:55:50.875476 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:50.875444    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 21:55:50.914472 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:50.914445    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 21:55:55.369474 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.369440    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:55:55.373199 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.373180    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.378436 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.378408    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-9e9409-dockercfg-r6z88\""
Sep 17 21:55:55.380982 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.380964    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-9e9409\""
Sep 17 21:55:55.424649 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.424619    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:55:55.454128 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.454097    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.454311 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.454165    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hprk\" (UniqueName: \"kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.555731 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.555696    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.555878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.555761    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hprk\" (UniqueName: \"kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.556426 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.556407    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.580531 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.580498    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hprk\" (UniqueName: \"kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk\") pod \"payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.683739 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.683628    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:55.985978 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:55.985945    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:55:55.990016 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:55:55.989989    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c96e1b0_632d_46cd_863d_6f4fe2aa5a49.slice/crio-984639a588205b252ece36a2f612bdcf4e217b6f0547826f50dc6e2d87a4a3d4 WatchSource:0}: Error finding container 984639a588205b252ece36a2f612bdcf4e217b6f0547826f50dc6e2d87a4a3d4: Status 404 returned error can't find the container with id 984639a588205b252ece36a2f612bdcf4e217b6f0547826f50dc6e2d87a4a3d4
Sep 17 21:55:56.916500 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:56.916456    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 21:55:56.970263 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:56.970231    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" event={"ID":"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49","Type":"ContainerStarted","Data":"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"}
Sep 17 21:55:56.970453 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:56.970308    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" event={"ID":"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49","Type":"ContainerStarted","Data":"984639a588205b252ece36a2f612bdcf4e217b6f0547826f50dc6e2d87a4a3d4"}
Sep 17 21:55:56.970453 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:56.970340    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:55:57.135926 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:55:57.135865    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" podStartSLOduration=2.135847963 podStartE2EDuration="2.135847963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:55:57.134580657 +0000 UTC m=+860.483314538" watchObservedRunningTime="2026-09-17 21:55:57.135847963 +0000 UTC m=+860.484581842"
Sep 17 21:56:04.084194 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.084165    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:04.088116 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.088091    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.095727 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.095699    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4b89cb58-serving-cert\""
Sep 17 21:56:04.128268 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.128237    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:04.244400 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.244358    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.244400 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.244397    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zclrn\" (UniqueName: \"kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.244646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.244505    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.244646 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.244570    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.345552 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.345466    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.345552 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.345513    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.345750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.345578    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.345750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.345598    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zclrn\" (UniqueName: \"kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.346423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.346334    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.348166 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.348144    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.348242 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.348165    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.370302 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.370263    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zclrn\" (UniqueName: \"kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn\") pod \"maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.399312 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.399254    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:04.731184 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:04.731161    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:04.733419 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:56:04.733393    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85535744_0399_45d9_84b5_da2245f57519.slice/crio-0e21b6b0f555f3bec28e7aa9b2748242baf74e8be2ea1298cc22721d10fc8423 WatchSource:0}: Error finding container 0e21b6b0f555f3bec28e7aa9b2748242baf74e8be2ea1298cc22721d10fc8423: Status 404 returned error can't find the container with id 0e21b6b0f555f3bec28e7aa9b2748242baf74e8be2ea1298cc22721d10fc8423
Sep 17 21:56:05.023203 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:05.023119    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" event={"ID":"85535744-0399-45d9-84b5-da2245f57519","Type":"ContainerStarted","Data":"0e21b6b0f555f3bec28e7aa9b2748242baf74e8be2ea1298cc22721d10fc8423"}
Sep 17 21:56:06.030265 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:06.030220    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" event={"ID":"85535744-0399-45d9-84b5-da2245f57519","Type":"ContainerStarted","Data":"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"}
Sep 17 21:56:06.030773 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:06.030277    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:06.111754 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:06.111687    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" podStartSLOduration=1.702344012 podStartE2EDuration="2.111666607s" totalImagesPullingTime="409.322595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:56:06.109429649 +0000 UTC m=+869.458163532" watchObservedRunningTime="2026-09-17 21:56:06.111666607 +0000 UTC m=+869.460400489"
Sep 17 21:56:07.977161 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:07.977130    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:56:12.042320 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:12.042266    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:13.020009 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.019969    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 21:56:13.023985 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.023958    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.028767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.028738    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3d5c73\""
Sep 17 21:56:13.028767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.028762    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3d5c73-dockercfg-4bzlt\""
Sep 17 21:56:13.053797 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.053765    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 21:56:13.136005 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.135969    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.136149 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.136032    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrc7m\" (UniqueName: \"kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.237089 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.237045    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.237277 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.237133    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrc7m\" (UniqueName: \"kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.237770 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.237748    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.268072 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.268039    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrc7m\" (UniqueName: \"kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m\") pod \"payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.358318 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.358181    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:13.670899 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:13.670870    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 21:56:13.672559 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:56:13.672532    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36bef8f_a2bd_4228_897b_a3bcdda89431.slice/crio-b60146de0f1685a7a711fc12e96f940b7f5a1b308c598acd480ff2608cc9474b WatchSource:0}: Error finding container b60146de0f1685a7a711fc12e96f940b7f5a1b308c598acd480ff2608cc9474b: Status 404 returned error can't find the container with id b60146de0f1685a7a711fc12e96f940b7f5a1b308c598acd480ff2608cc9474b
Sep 17 21:56:14.080540 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:14.080497    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" event={"ID":"f36bef8f-a2bd-4228-897b-a3bcdda89431","Type":"ContainerStarted","Data":"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"}
Sep 17 21:56:14.080983 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:14.080562    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" event={"ID":"f36bef8f-a2bd-4228-897b-a3bcdda89431","Type":"ContainerStarted","Data":"b60146de0f1685a7a711fc12e96f940b7f5a1b308c598acd480ff2608cc9474b"}
Sep 17 21:56:14.080983 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:14.080644    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:14.126706 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:14.126649    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" podStartSLOduration=2.126632148 podStartE2EDuration="2.126632148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:56:14.12378308 +0000 UTC m=+877.472516962" watchObservedRunningTime="2026-09-17 21:56:14.126632148 +0000 UTC m=+877.475366031"
Sep 17 21:56:25.088200 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:25.088168    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 21:56:35.822519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:35.822486    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:35.822986 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:35.822756    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" podUID="85535744-0399-45d9-84b5-da2245f57519" containerName="maas-api" containerID="cri-o://1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4" gracePeriod=30
Sep 17 21:56:36.069350 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.069320    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:36.158423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158351    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs\") pod \"85535744-0399-45d9-84b5-da2245f57519\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") "
Sep 17 21:56:36.158423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158401    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca\") pod \"85535744-0399-45d9-84b5-da2245f57519\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") "
Sep 17 21:56:36.158662 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158425    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zclrn\" (UniqueName: \"kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn\") pod \"85535744-0399-45d9-84b5-da2245f57519\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") "
Sep 17 21:56:36.158662 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158459    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls\") pod \"85535744-0399-45d9-84b5-da2245f57519\" (UID: \"85535744-0399-45d9-84b5-da2245f57519\") "
Sep 17 21:56:36.158780 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158763    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85535744-0399-45d9-84b5-da2245f57519" (UID: "85535744-0399-45d9-84b5-da2245f57519"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:56:36.158890 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.158872    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85535744-0399-45d9-84b5-da2245f57519-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:56:36.160831 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.160805    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85535744-0399-45d9-84b5-da2245f57519" (UID: "85535744-0399-45d9-84b5-da2245f57519"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:56:36.160939 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.160849    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "85535744-0399-45d9-84b5-da2245f57519" (UID: "85535744-0399-45d9-84b5-da2245f57519"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:56:36.160939 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.160866    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn" (OuterVolumeSpecName: "kube-api-access-zclrn") pod "85535744-0399-45d9-84b5-da2245f57519" (UID: "85535744-0399-45d9-84b5-da2245f57519"). InnerVolumeSpecName "kube-api-access-zclrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:56:36.204813 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.204777    2587 generic.go:356] "Generic (PLEG): container finished" podID="85535744-0399-45d9-84b5-da2245f57519" containerID="1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4" exitCode=0
Sep 17 21:56:36.204951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.204828    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"
Sep 17 21:56:36.204951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.204858    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" event={"ID":"85535744-0399-45d9-84b5-da2245f57519","Type":"ContainerDied","Data":"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"}
Sep 17 21:56:36.204951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.204910    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx" event={"ID":"85535744-0399-45d9-84b5-da2245f57519","Type":"ContainerDied","Data":"0e21b6b0f555f3bec28e7aa9b2748242baf74e8be2ea1298cc22721d10fc8423"}
Sep 17 21:56:36.204951 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.204933    2587 scope.go:117] "RemoveContainer" containerID="1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"
Sep 17 21:56:36.215423 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.215402    2587 scope.go:117] "RemoveContainer" containerID="1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"
Sep 17 21:56:36.215723 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:56:36.215695    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4\": container with ID starting with 1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4 not found: ID does not exist" containerID="1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"
Sep 17 21:56:36.215820 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.215729    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4"} err="failed to get container status \"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4\": rpc error: code = NotFound desc = could not find container \"1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4\": container with ID starting with 1f4133ba7a6ce78d5fcf4dab08d14ca2acb26f280b67832c443c0a56646b77e4 not found: ID does not exist"
Sep 17 21:56:36.242720 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.242441    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:36.251644 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.251584    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b89cb58-578bf5bf9-zxvsx"]
Sep 17 21:56:36.259575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.259546    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:56:36.259575 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.259570    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zclrn\" (UniqueName: \"kubernetes.io/projected/85535744-0399-45d9-84b5-da2245f57519-kube-api-access-zclrn\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:56:36.259719 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.259580    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85535744-0399-45d9-84b5-da2245f57519-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:56:36.857250 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:36.857216    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85535744-0399-45d9-84b5-da2245f57519" path="/var/lib/kubelet/pods/85535744-0399-45d9-84b5-da2245f57519/volumes"
Sep 17 21:56:38.801964 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:38.801929    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:56:38.803755 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:38.803735    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 21:56:44.857543 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.857512    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:56:44.857952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.857931    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85535744-0399-45d9-84b5-da2245f57519" containerName="maas-api"
Sep 17 21:56:44.857952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.857941    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="85535744-0399-45d9-84b5-da2245f57519" containerName="maas-api"
Sep 17 21:56:44.858039 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.858005    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="85535744-0399-45d9-84b5-da2245f57519" containerName="maas-api"
Sep 17 21:56:44.862420 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.862401    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:44.866249 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.866224    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a0a97fdf-dockercfg-mrmwp\""
Sep 17 21:56:44.866368 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.866343    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a0a97fdf\""
Sep 17 21:56:44.874860 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.874834    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:56:44.936455 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.936427    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:56:44.940322 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.940278    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:44.944081 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.944064    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6wsk\" (UniqueName: \"kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:44.944158 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.944130    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:44.957703 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:44.957676    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:56:45.044899 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.044866    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.045048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.044909    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6wsk\" (UniqueName: \"kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:45.045048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.044935    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kbg\" (UniqueName: \"kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.045048 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.045013    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:45.045833 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.045810    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:45.054573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.054549    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6wsk\" (UniqueName: \"kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk\") pod \"payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:45.145904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.145810    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.145904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.145876    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kbg\" (UniqueName: \"kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.146590 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.146569    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.157924 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.157895    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kbg\" (UniqueName: \"kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg\") pod \"payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") " pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.172717 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.172692    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:45.249881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.249851    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:45.459036 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.458883    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:56:45.461856 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:56:45.461836    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9170ddc1_2976_43ac_a3fe_55b467e6709e.slice/crio-60692abf6f6154184905d4aec8eecfaf4e88f2aae0ebcf726e1bb3bd31dd0296 WatchSource:0}: Error finding container 60692abf6f6154184905d4aec8eecfaf4e88f2aae0ebcf726e1bb3bd31dd0296: Status 404 returned error can't find the container with id 60692abf6f6154184905d4aec8eecfaf4e88f2aae0ebcf726e1bb3bd31dd0296
Sep 17 21:56:45.531167 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:45.531141    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:56:45.532906 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:56:45.532882    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd770f8a0_09c5_4504_b0af_a62ceb09a40a.slice/crio-ca253e8c53e9c728b0b3f260e4c73f9db5b35ac96aa37f937e5e409add088672 WatchSource:0}: Error finding container ca253e8c53e9c728b0b3f260e4c73f9db5b35ac96aa37f937e5e409add088672: Status 404 returned error can't find the container with id ca253e8c53e9c728b0b3f260e4c73f9db5b35ac96aa37f937e5e409add088672
Sep 17 21:56:46.262175 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.262133    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" event={"ID":"9170ddc1-2976-43ac-a3fe-55b467e6709e","Type":"ContainerStarted","Data":"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"}
Sep 17 21:56:46.262682 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.262194    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" event={"ID":"9170ddc1-2976-43ac-a3fe-55b467e6709e","Type":"ContainerStarted","Data":"60692abf6f6154184905d4aec8eecfaf4e88f2aae0ebcf726e1bb3bd31dd0296"}
Sep 17 21:56:46.262682 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.262220    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:46.264039 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.264013    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" event={"ID":"d770f8a0-09c5-4504-b0af-a62ceb09a40a","Type":"ContainerStarted","Data":"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"}
Sep 17 21:56:46.264159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.264059    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" event={"ID":"d770f8a0-09c5-4504-b0af-a62ceb09a40a","Type":"ContainerStarted","Data":"ca253e8c53e9c728b0b3f260e4c73f9db5b35ac96aa37f937e5e409add088672"}
Sep 17 21:56:46.264159 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.264083    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:56:46.291963 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.291911    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" podStartSLOduration=2.291892271 podStartE2EDuration="2.291892271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:56:46.29159 +0000 UTC m=+909.640323888" watchObservedRunningTime="2026-09-17 21:56:46.291892271 +0000 UTC m=+909.640626152"
Sep 17 21:56:46.314842 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:46.314793    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" podStartSLOduration=2.314780269 podStartE2EDuration="2.314780269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:56:46.313947625 +0000 UTC m=+909.662681507" watchObservedRunningTime="2026-09-17 21:56:46.314780269 +0000 UTC m=+909.663514148"
Sep 17 21:56:57.271192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:57.271161    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:56:57.271728 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:56:57.271359    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:57:19.768813 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.768778    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:57:19.772404 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.772384    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.775591 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.775570    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9e9409\"/\"kube-root-ca.crt\""
Sep 17 21:57:19.776380 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.776363    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9e9409\"/\"default-dockercfg-9t9tw\""
Sep 17 21:57:19.776597 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.776582    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9e9409\"/\"autores-model-9e9409-kserve-self-signed-certs\""
Sep 17 21:57:19.776672 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.776621    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9e9409\"/\"openshift-service-ca.crt\""
Sep 17 21:57:19.787239 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.787214    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:57:19.891522 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891483    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.891746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891535    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slxgq\" (UniqueName: \"kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.891746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891601    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.891746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891656    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.891746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891676    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.891746 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.891741    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992788 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992744    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slxgq\" (UniqueName: \"kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992802    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992839    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992856    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992896    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.992980 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.992956    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.993377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.993345    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.993377 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.993361    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.993522 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.993394    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.995464 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.995433    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:19.995745 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:19.995726    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:20.002860 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.002829    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slxgq\" (UniqueName: \"kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq\") pod \"autores-model-9e9409-kserve-6b64759899-kgndg\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") " pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:20.082622 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.082504    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:20.360181 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.360148    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:57:20.361191 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:57:20.361165    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod042b1610_5ace_460c_8a71_ab4d89fdfdd1.slice/crio-7ec33758d90dc7e93c57c78954cd78968aba129b6e4e48e14492743d87043882 WatchSource:0}: Error finding container 7ec33758d90dc7e93c57c78954cd78968aba129b6e4e48e14492743d87043882: Status 404 returned error can't find the container with id 7ec33758d90dc7e93c57c78954cd78968aba129b6e4e48e14492743d87043882
Sep 17 21:57:20.451391 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.451358    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" event={"ID":"042b1610-5ace-460c-8a71-ab4d89fdfdd1","Type":"ContainerStarted","Data":"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"}
Sep 17 21:57:20.451591 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.451402    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" event={"ID":"042b1610-5ace-460c-8a71-ab4d89fdfdd1","Type":"ContainerStarted","Data":"7ec33758d90dc7e93c57c78954cd78968aba129b6e4e48e14492743d87043882"}
Sep 17 21:57:20.497508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:20.497450    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" podStartSLOduration=1.497433448 podStartE2EDuration="1.497433448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:57:20.495745629 +0000 UTC m=+943.844479513" watchObservedRunningTime="2026-09-17 21:57:20.497433448 +0000 UTC m=+943.846167368"
Sep 17 21:57:22.643601 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.643570    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:57:22.644069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.643843    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" podUID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" containerName="payload-processing" containerID="cri-o://f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4" gracePeriod=30
Sep 17 21:57:22.660595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.660566    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:57:22.660835 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.660807    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" podUID="9170ddc1-2976-43ac-a3fe-55b467e6709e" containerName="payload-pre-processing" containerID="cri-o://6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8" gracePeriod=30
Sep 17 21:57:22.942649 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.942624    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:57:22.945665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:22.945647    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:57:23.027264 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027231    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume\") pod \"9170ddc1-2976-43ac-a3fe-55b467e6709e\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") "
Sep 17 21:57:23.027458 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027296    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume\") pod \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") "
Sep 17 21:57:23.027458 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027365    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2kbg\" (UniqueName: \"kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg\") pod \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\" (UID: \"d770f8a0-09c5-4504-b0af-a62ceb09a40a\") "
Sep 17 21:57:23.027458 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027441    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6wsk\" (UniqueName: \"kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk\") pod \"9170ddc1-2976-43ac-a3fe-55b467e6709e\" (UID: \"9170ddc1-2976-43ac-a3fe-55b467e6709e\") "
Sep 17 21:57:23.027732 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027704    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9170ddc1-2976-43ac-a3fe-55b467e6709e" (UID: "9170ddc1-2976-43ac-a3fe-55b467e6709e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:57:23.027796 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.027768    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d770f8a0-09c5-4504-b0af-a62ceb09a40a" (UID: "d770f8a0-09c5-4504-b0af-a62ceb09a40a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:57:23.029935 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.029908    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg" (OuterVolumeSpecName: "kube-api-access-z2kbg") pod "d770f8a0-09c5-4504-b0af-a62ceb09a40a" (UID: "d770f8a0-09c5-4504-b0af-a62ceb09a40a"). InnerVolumeSpecName "kube-api-access-z2kbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:57:23.030064 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.029930    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk" (OuterVolumeSpecName: "kube-api-access-p6wsk") pod "9170ddc1-2976-43ac-a3fe-55b467e6709e" (UID: "9170ddc1-2976-43ac-a3fe-55b467e6709e"). InnerVolumeSpecName "kube-api-access-p6wsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:57:23.128545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.128503    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2kbg\" (UniqueName: \"kubernetes.io/projected/d770f8a0-09c5-4504-b0af-a62ceb09a40a-kube-api-access-z2kbg\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:57:23.128545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.128549    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6wsk\" (UniqueName: \"kubernetes.io/projected/9170ddc1-2976-43ac-a3fe-55b467e6709e-kube-api-access-p6wsk\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:57:23.128867 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.128565    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9170ddc1-2976-43ac-a3fe-55b467e6709e-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:57:23.128867 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.128580    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d770f8a0-09c5-4504-b0af-a62ceb09a40a-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:57:23.469582 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.469551    2587 generic.go:356] "Generic (PLEG): container finished" podID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" containerID="f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4" exitCode=0
Sep 17 21:57:23.469767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.469593    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"
Sep 17 21:57:23.469767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.469640    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" event={"ID":"d770f8a0-09c5-4504-b0af-a62ceb09a40a","Type":"ContainerDied","Data":"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"}
Sep 17 21:57:23.469767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.469688    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt" event={"ID":"d770f8a0-09c5-4504-b0af-a62ceb09a40a","Type":"ContainerDied","Data":"ca253e8c53e9c728b0b3f260e4c73f9db5b35ac96aa37f937e5e409add088672"}
Sep 17 21:57:23.469767 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.469712    2587 scope.go:117] "RemoveContainer" containerID="f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"
Sep 17 21:57:23.471196 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.471170    2587 generic.go:356] "Generic (PLEG): container finished" podID="9170ddc1-2976-43ac-a3fe-55b467e6709e" containerID="6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8" exitCode=0
Sep 17 21:57:23.471360 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.471223    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"
Sep 17 21:57:23.471360 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.471236    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" event={"ID":"9170ddc1-2976-43ac-a3fe-55b467e6709e","Type":"ContainerDied","Data":"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"}
Sep 17 21:57:23.471360 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.471277    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr" event={"ID":"9170ddc1-2976-43ac-a3fe-55b467e6709e","Type":"ContainerDied","Data":"60692abf6f6154184905d4aec8eecfaf4e88f2aae0ebcf726e1bb3bd31dd0296"}
Sep 17 21:57:23.481508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.481484    2587 scope.go:117] "RemoveContainer" containerID="f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"
Sep 17 21:57:23.481809 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:57:23.481781    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4\": container with ID starting with f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4 not found: ID does not exist" containerID="f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"
Sep 17 21:57:23.481925 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.481817    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4"} err="failed to get container status \"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4\": rpc error: code = NotFound desc = could not find container \"f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4\": container with ID starting with f298457797febf783cc7149ea2bdac7afa9e68186c1fca9157f6c788cf892dc4 not found: ID does not exist"
Sep 17 21:57:23.481925 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.481836    2587 scope.go:117] "RemoveContainer" containerID="6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"
Sep 17 21:57:23.490871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.490854    2587 scope.go:117] "RemoveContainer" containerID="6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"
Sep 17 21:57:23.491166 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:57:23.491144    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8\": container with ID starting with 6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8 not found: ID does not exist" containerID="6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"
Sep 17 21:57:23.491236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.491173    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8"} err="failed to get container status \"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8\": rpc error: code = NotFound desc = could not find container \"6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8\": container with ID starting with 6fe8571b3263fc9e2a22e18c3af06c5418171a1c87f75e92169c163f05673ad8 not found: ID does not exist"
Sep 17 21:57:23.509801 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.509776    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:57:23.518823 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.518801    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a0a97fdf-544d64f5b6-pqcjt"]
Sep 17 21:57:23.557601 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.557570    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:57:23.575061 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:23.575023    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a0a97fdf-85547b76f8-tbwsr"]
Sep 17 21:57:24.857520 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:24.857476    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9170ddc1-2976-43ac-a3fe-55b467e6709e" path="/var/lib/kubelet/pods/9170ddc1-2976-43ac-a3fe-55b467e6709e/volumes"
Sep 17 21:57:24.858035 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:24.858013    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" path="/var/lib/kubelet/pods/d770f8a0-09c5-4504-b0af-a62ceb09a40a/volumes"
Sep 17 21:57:30.083568 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:30.083523    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:30.084043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:30.083585    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:30.096480 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:30.096449    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:30.520344 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:30.520316    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:57:36.457508 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.457476    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:57:36.457967 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.457950    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9170ddc1-2976-43ac-a3fe-55b467e6709e" containerName="payload-pre-processing"
Sep 17 21:57:36.458013 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.457970    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9170ddc1-2976-43ac-a3fe-55b467e6709e" containerName="payload-pre-processing"
Sep 17 21:57:36.458013 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.457991    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" containerName="payload-processing"
Sep 17 21:57:36.458013 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.457997    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" containerName="payload-processing"
Sep 17 21:57:36.458169 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.458071    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d770f8a0-09c5-4504-b0af-a62ceb09a40a" containerName="payload-processing"
Sep 17 21:57:36.458169 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.458087    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9170ddc1-2976-43ac-a3fe-55b467e6709e" containerName="payload-pre-processing"
Sep 17 21:57:36.462719 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.462703    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.468132 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.468113    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a9151699-dockercfg-sk9vt\""
Sep 17 21:57:36.468654 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.468634    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a9151699\""
Sep 17 21:57:36.494184 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.494159    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:57:36.516639 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.516609    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:57:36.535585 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.535555    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.558687 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.558656    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:57:36.563344 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.563317    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z48j5\" (UniqueName: \"kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.563461 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.563444    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.664707 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.664670    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z48j5\" (UniqueName: \"kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.664859 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.664720    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drf5n\" (UniqueName: \"kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.664859 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.664769    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.664859 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.664798    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.665484 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.665463    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.684363 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.684339    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z48j5\" (UniqueName: \"kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5\") pod \"payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.766152 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.766120    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drf5n\" (UniqueName: \"kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.766370 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.766185    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.766870 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.766849    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.773016 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.772983    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:36.782588 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.782564    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drf5n\" (UniqueName: \"kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n\") pod \"payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") " pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:36.847748 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:36.847710    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:37.054741 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.054703    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:57:37.055296 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:57:37.055257    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9ca1fe_f962_47b2_a5c8_6a75a636fa07.slice/crio-0f8531cba10e873be5cebd31ed6114184cba158a2871140e2f70b095cc5ee805 WatchSource:0}: Error finding container 0f8531cba10e873be5cebd31ed6114184cba158a2871140e2f70b095cc5ee805: Status 404 returned error can't find the container with id 0f8531cba10e873be5cebd31ed6114184cba158a2871140e2f70b095cc5ee805
Sep 17 21:57:37.151562 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.151537    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:57:37.153588 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:57:37.153564    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4adeed7_7f26_4cac_a3d7_c3bf5e325bf4.slice/crio-d8b84b52bcbf6e3a8ab57919a8764fb5c33a05656093b23fa2b01924d5016ec1 WatchSource:0}: Error finding container d8b84b52bcbf6e3a8ab57919a8764fb5c33a05656093b23fa2b01924d5016ec1: Status 404 returned error can't find the container with id d8b84b52bcbf6e3a8ab57919a8764fb5c33a05656093b23fa2b01924d5016ec1
Sep 17 21:57:37.548781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.548742    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" event={"ID":"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4","Type":"ContainerStarted","Data":"7c7ceefb048681e19fed7dbf6e6f925030520ab5877d25265050ecca8cb2ec2a"}
Sep 17 21:57:37.549275 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.548801    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" event={"ID":"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4","Type":"ContainerStarted","Data":"d8b84b52bcbf6e3a8ab57919a8764fb5c33a05656093b23fa2b01924d5016ec1"}
Sep 17 21:57:37.549275 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.548948    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:37.550428 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.550406    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" event={"ID":"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07","Type":"ContainerStarted","Data":"6403d46019e048b134a5b06d4fdf88b52269cb91cc49e0a4b93e014c6bb197ea"}
Sep 17 21:57:37.550535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.550447    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" event={"ID":"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07","Type":"ContainerStarted","Data":"0f8531cba10e873be5cebd31ed6114184cba158a2871140e2f70b095cc5ee805"}
Sep 17 21:57:37.550535 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.550466    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:37.584144 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.584086    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" podStartSLOduration=1.584072854 podStartE2EDuration="1.584072854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:57:37.580737678 +0000 UTC m=+960.929471557" watchObservedRunningTime="2026-09-17 21:57:37.584072854 +0000 UTC m=+960.932806773"
Sep 17 21:57:37.616005 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:37.615953    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" podStartSLOduration=1.615938576 podStartE2EDuration="1.615938576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:57:37.613886197 +0000 UTC m=+960.962620076" watchObservedRunningTime="2026-09-17 21:57:37.615938576 +0000 UTC m=+960.964672456"
Sep 17 21:57:48.558280 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:48.558249    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:57:48.558764 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:48.558427    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:57:57.289629 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:57:57.289597    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:58:07.355521 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.355479    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:58:07.356429 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.356390    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" podUID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" containerName="payload-pre-processing" containerID="cri-o://6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39" gracePeriod=30
Sep 17 21:58:07.622429 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.622363    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:58:07.715712 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.715677    2587 generic.go:356] "Generic (PLEG): container finished" podID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" containerID="6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39" exitCode=0
Sep 17 21:58:07.715878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.715725    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"
Sep 17 21:58:07.715878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.715766    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" event={"ID":"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49","Type":"ContainerDied","Data":"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"}
Sep 17 21:58:07.715878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.715823    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr" event={"ID":"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49","Type":"ContainerDied","Data":"984639a588205b252ece36a2f612bdcf4e217b6f0547826f50dc6e2d87a4a3d4"}
Sep 17 21:58:07.715878 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.715851    2587 scope.go:117] "RemoveContainer" containerID="6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"
Sep 17 21:58:07.724667 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.724651    2587 scope.go:117] "RemoveContainer" containerID="6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"
Sep 17 21:58:07.724940 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:58:07.724920    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39\": container with ID starting with 6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39 not found: ID does not exist" containerID="6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"
Sep 17 21:58:07.724987 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.724951    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39"} err="failed to get container status \"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39\": rpc error: code = NotFound desc = could not find container \"6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39\": container with ID starting with 6d85f720993a61d58de09ef3a38e8273334564f728914004cd57a3c3ccdfca39 not found: ID does not exist"
Sep 17 21:58:07.773469 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.773434    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume\") pod \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") "
Sep 17 21:58:07.773469 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.773473    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hprk\" (UniqueName: \"kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk\") pod \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\" (UID: \"4c96e1b0-632d-46cd-863d-6f4fe2aa5a49\") "
Sep 17 21:58:07.773851 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.773831    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" (UID: "4c96e1b0-632d-46cd-863d-6f4fe2aa5a49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:07.775789 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.775757    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk" (OuterVolumeSpecName: "kube-api-access-4hprk") pod "4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" (UID: "4c96e1b0-632d-46cd-863d-6f4fe2aa5a49"). InnerVolumeSpecName "kube-api-access-4hprk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:07.874595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.874511    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:07.874595 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:07.874541    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hprk\" (UniqueName: \"kubernetes.io/projected/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49-kube-api-access-4hprk\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:08.040019 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:08.039989    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:58:08.047248 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:08.047222    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9e9409-cfcfff89d-q6srr"]
Sep 17 21:58:08.857359 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:08.857323    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" path="/var/lib/kubelet/pods/4c96e1b0-632d-46cd-863d-6f4fe2aa5a49/volumes"
Sep 17 21:58:12.309261 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.309223    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" podUID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" containerName="main" containerID="cri-o://20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724" gracePeriod=45
Sep 17 21:58:12.567069 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.567009    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:58:12.725232 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725193    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725232 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725234    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725271    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slxgq\" (UniqueName: \"kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725336    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725406    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725537 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725485    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs\") pod \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\" (UID: \"042b1610-5ace-460c-8a71-ab4d89fdfdd1\") "
Sep 17 21:58:12.725761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725534    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:58:12.725761 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725602    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home" (OuterVolumeSpecName: "home") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:58:12.725879 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725852    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache" (OuterVolumeSpecName: "model-cache") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:58:12.726020 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725891    2587 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tmp-dir\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.726020 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.725914    2587 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-home\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.727904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.727880    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm" (OuterVolumeSpecName: "dshm") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:58:12.727904 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.727893    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:58:12.728024 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.727988    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq" (OuterVolumeSpecName: "kube-api-access-slxgq") pod "042b1610-5ace-460c-8a71-ab4d89fdfdd1" (UID: "042b1610-5ace-460c-8a71-ab4d89fdfdd1"). InnerVolumeSpecName "kube-api-access-slxgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:12.744388 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.744350    2587 generic.go:356] "Generic (PLEG): container finished" podID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" containerID="20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724" exitCode=0
Sep 17 21:58:12.744593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.744388    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" event={"ID":"042b1610-5ace-460c-8a71-ab4d89fdfdd1","Type":"ContainerDied","Data":"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"}
Sep 17 21:58:12.744593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.744410    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"
Sep 17 21:58:12.744593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.744448    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg" event={"ID":"042b1610-5ace-460c-8a71-ab4d89fdfdd1","Type":"ContainerDied","Data":"7ec33758d90dc7e93c57c78954cd78968aba129b6e4e48e14492743d87043882"}
Sep 17 21:58:12.744593 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.744471    2587 scope.go:117] "RemoveContainer" containerID="20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"
Sep 17 21:58:12.754308 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.754275    2587 scope.go:117] "RemoveContainer" containerID="20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"
Sep 17 21:58:12.754666 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:58:12.754647    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724\": container with ID starting with 20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724 not found: ID does not exist" containerID="20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"
Sep 17 21:58:12.754716 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.754675    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724"} err="failed to get container status \"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724\": rpc error: code = NotFound desc = could not find container \"20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724\": container with ID starting with 20f0d3991d719eb5b3231fcfa9908672df32639654022dff739f59b1643ea724 not found: ID does not exist"
Sep 17 21:58:12.771836 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.771787    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:58:12.774521 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.774492    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-9e9409/autores-model-9e9409-kserve-6b64759899-kgndg"]
Sep 17 21:58:12.826912 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.826819    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slxgq\" (UniqueName: \"kubernetes.io/projected/042b1610-5ace-460c-8a71-ab4d89fdfdd1-kube-api-access-slxgq\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.826912 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.826851    2587 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-dshm\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.826912 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.826862    2587 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/042b1610-5ace-460c-8a71-ab4d89fdfdd1-model-cache\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.826912 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.826872    2587 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/042b1610-5ace-460c-8a71-ab4d89fdfdd1-tls-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:12.857042 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:12.857006    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" path="/var/lib/kubelet/pods/042b1610-5ace-460c-8a71-ab4d89fdfdd1/volumes"
Sep 17 21:58:14.514581 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.514549    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:58:14.515009 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.514774    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" podUID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" containerName="payload-processing" containerID="cri-o://7c7ceefb048681e19fed7dbf6e6f925030520ab5877d25265050ecca8cb2ec2a" gracePeriod=30
Sep 17 21:58:14.535627 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.535598    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:58:14.535871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.535822    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" podUID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" containerName="payload-pre-processing" containerID="cri-o://6403d46019e048b134a5b06d4fdf88b52269cb91cc49e0a4b93e014c6bb197ea" gracePeriod=30
Sep 17 21:58:14.759882 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.759857    2587 generic.go:356] "Generic (PLEG): container finished" podID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" containerID="7c7ceefb048681e19fed7dbf6e6f925030520ab5877d25265050ecca8cb2ec2a" exitCode=0
Sep 17 21:58:14.760017 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.759928    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" event={"ID":"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4","Type":"ContainerDied","Data":"7c7ceefb048681e19fed7dbf6e6f925030520ab5877d25265050ecca8cb2ec2a"}
Sep 17 21:58:14.761885 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.761856    2587 generic.go:356] "Generic (PLEG): container finished" podID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" containerID="6403d46019e048b134a5b06d4fdf88b52269cb91cc49e0a4b93e014c6bb197ea" exitCode=0
Sep 17 21:58:14.762021 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.761931    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" event={"ID":"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07","Type":"ContainerDied","Data":"6403d46019e048b134a5b06d4fdf88b52269cb91cc49e0a4b93e014c6bb197ea"}
Sep 17 21:58:14.788769 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.788744    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:58:14.817461 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.817436    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:58:14.950146 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950111    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume\") pod \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") "
Sep 17 21:58:14.950146 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950148    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z48j5\" (UniqueName: \"kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5\") pod \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\" (UID: \"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07\") "
Sep 17 21:58:14.950388 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950190    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drf5n\" (UniqueName: \"kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n\") pod \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") "
Sep 17 21:58:14.950388 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950251    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume\") pod \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\" (UID: \"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4\") "
Sep 17 21:58:14.950622 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950583    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" (UID: "1d9ca1fe-f962-47b2-a5c8-6a75a636fa07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:14.950816 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.950670    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" (UID: "d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:14.952570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.952538    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5" (OuterVolumeSpecName: "kube-api-access-z48j5") pod "1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" (UID: "1d9ca1fe-f962-47b2-a5c8-6a75a636fa07"). InnerVolumeSpecName "kube-api-access-z48j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:14.952661 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:14.952632    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n" (OuterVolumeSpecName: "kube-api-access-drf5n") pod "d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" (UID: "d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4"). InnerVolumeSpecName "kube-api-access-drf5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:15.051688 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.051611    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:15.051688 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.051640    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z48j5\" (UniqueName: \"kubernetes.io/projected/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07-kube-api-access-z48j5\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:15.051688 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.051649    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drf5n\" (UniqueName: \"kubernetes.io/projected/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-kube-api-access-drf5n\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:15.051688 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.051658    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:15.768076 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.768049    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"
Sep 17 21:58:15.768551 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.768038    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6" event={"ID":"d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4","Type":"ContainerDied","Data":"d8b84b52bcbf6e3a8ab57919a8764fb5c33a05656093b23fa2b01924d5016ec1"}
Sep 17 21:58:15.768551 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.768211    2587 scope.go:117] "RemoveContainer" containerID="7c7ceefb048681e19fed7dbf6e6f925030520ab5877d25265050ecca8cb2ec2a"
Sep 17 21:58:15.769690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.769433    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"
Sep 17 21:58:15.769690 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.769437    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww" event={"ID":"1d9ca1fe-f962-47b2-a5c8-6a75a636fa07","Type":"ContainerDied","Data":"0f8531cba10e873be5cebd31ed6114184cba158a2871140e2f70b095cc5ee805"}
Sep 17 21:58:15.779174 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.779146    2587 scope.go:117] "RemoveContainer" containerID="6403d46019e048b134a5b06d4fdf88b52269cb91cc49e0a4b93e014c6bb197ea"
Sep 17 21:58:15.814417 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.814379    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:58:15.821523 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.821494    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9151699-6d5bdd5dc4-hswww"]
Sep 17 21:58:15.838489 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.838458    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:58:15.845180 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:15.845154    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9151699-9c5d89f97-52fh6"]
Sep 17 21:58:16.858043 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:16.857960    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" path="/var/lib/kubelet/pods/1d9ca1fe-f962-47b2-a5c8-6a75a636fa07/volumes"
Sep 17 21:58:16.858439 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:16.858359    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" path="/var/lib/kubelet/pods/d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4/volumes"
Sep 17 21:58:27.868439 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868406    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868859    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" containerName="payload-pre-processing"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868870    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" containerName="payload-pre-processing"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868885    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" containerName="payload-processing"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868891    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" containerName="payload-processing"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868898    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" containerName="main"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868903    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" containerName="main"
Sep 17 21:58:27.868915 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868917    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" containerName="payload-pre-processing"
Sep 17 21:58:27.869141 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868922    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" containerName="payload-pre-processing"
Sep 17 21:58:27.869141 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868986    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d9ca1fe-f962-47b2-a5c8-6a75a636fa07" containerName="payload-pre-processing"
Sep 17 21:58:27.869141 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.868993    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4adeed7-7f26-4cac-a3d7-c3bf5e325bf4" containerName="payload-processing"
Sep 17 21:58:27.869141 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.869000    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="042b1610-5ace-460c-8a71-ab4d89fdfdd1" containerName="main"
Sep 17 21:58:27.869141 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.869010    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c96e1b0-632d-46cd-863d-6f4fe2aa5a49" containerName="payload-pre-processing"
Sep 17 21:58:27.872192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.872176    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:27.877134 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.877114    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c85107f9\""
Sep 17 21:58:27.877134 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.877120    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c85107f9-dockercfg-2t9zf\""
Sep 17 21:58:27.900824 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.900793    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:27.978952 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.978916    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gbfn\" (UniqueName: \"kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:27.979150 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.978988    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:27.983918 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.983884    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:27.987521 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:27.987506    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.042562 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.042530    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:28.079783 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.079751    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gbfn\" (UniqueName: \"kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.079999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.079791    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.079999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.079851    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.079999 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.079888    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8995\" (UniqueName: \"kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.080549 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.080530    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.107573 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.107538    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gbfn\" (UniqueName: \"kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn\") pod \"payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.181091 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.180995    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.181091 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.181064    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8995\" (UniqueName: \"kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.181726 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.181705    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.182823 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.182797    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.198502 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.198475    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8995\" (UniqueName: \"kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995\") pod \"payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") " pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.297712 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.297668    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.497217 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.497194    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:28.499175 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:58:28.499148    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b475da_1067_429c_abdb_2294210bdf4d.slice/crio-72ce7e626aa1aae31a94f39359d1f0ea1713884604d6891b338f7ecab026a160 WatchSource:0}: Error finding container 72ce7e626aa1aae31a94f39359d1f0ea1713884604d6891b338f7ecab026a160: Status 404 returned error can't find the container with id 72ce7e626aa1aae31a94f39359d1f0ea1713884604d6891b338f7ecab026a160
Sep 17 21:58:28.626063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.626037    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:28.628850 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:58:28.628818    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099a8977_33e3_47e4_a09e_228b7920ade0.slice/crio-193071ce1bd6e1dd6f67fcc759beb103dc4dc26e127e8e8650a0f0cd15b1d09d WatchSource:0}: Error finding container 193071ce1bd6e1dd6f67fcc759beb103dc4dc26e127e8e8650a0f0cd15b1d09d: Status 404 returned error can't find the container with id 193071ce1bd6e1dd6f67fcc759beb103dc4dc26e127e8e8650a0f0cd15b1d09d
Sep 17 21:58:28.844875 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.844784    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" event={"ID":"099a8977-33e3-47e4-a09e-228b7920ade0","Type":"ContainerStarted","Data":"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"}
Sep 17 21:58:28.844875 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.844842    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" event={"ID":"099a8977-33e3-47e4-a09e-228b7920ade0","Type":"ContainerStarted","Data":"193071ce1bd6e1dd6f67fcc759beb103dc4dc26e127e8e8650a0f0cd15b1d09d"}
Sep 17 21:58:28.845103 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.844936    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:28.846570 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.846542    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" event={"ID":"d2b475da-1067-429c-abdb-2294210bdf4d","Type":"ContainerStarted","Data":"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"}
Sep 17 21:58:28.846708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.846586    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" event={"ID":"d2b475da-1067-429c-abdb-2294210bdf4d","Type":"ContainerStarted","Data":"72ce7e626aa1aae31a94f39359d1f0ea1713884604d6891b338f7ecab026a160"}
Sep 17 21:58:28.846708 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.846630    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:28.930409 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:28.930359    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" podStartSLOduration=1.930345304 podStartE2EDuration="1.930345304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:28.929360301 +0000 UTC m=+1012.278094182" watchObservedRunningTime="2026-09-17 21:58:28.930345304 +0000 UTC m=+1012.279079181"
Sep 17 21:58:29.009381 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:29.009329    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" podStartSLOduration=2.009311623 podStartE2EDuration="2.009311623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:29.006666662 +0000 UTC m=+1012.355400543" watchObservedRunningTime="2026-09-17 21:58:29.009311623 +0000 UTC m=+1012.358045503"
Sep 17 21:58:39.855549 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:39.855505    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:39.857136 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:39.857111    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:53.591195 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.591159    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:53.591665 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.591436    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" podUID="099a8977-33e3-47e4-a09e-228b7920ade0" containerName="payload-processing" containerID="cri-o://ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b" gracePeriod=30
Sep 17 21:58:53.609415 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.609378    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:53.609684 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.609636    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" podUID="d2b475da-1067-429c-abdb-2294210bdf4d" containerName="payload-pre-processing" containerID="cri-o://d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175" gracePeriod=30
Sep 17 21:58:53.880564 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.880525    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:53.883128 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.883111    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:53.990874 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.990844    2587 generic.go:356] "Generic (PLEG): container finished" podID="099a8977-33e3-47e4-a09e-228b7920ade0" containerID="ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b" exitCode=0
Sep 17 21:58:53.991063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.990901    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"
Sep 17 21:58:53.991063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.990926    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" event={"ID":"099a8977-33e3-47e4-a09e-228b7920ade0","Type":"ContainerDied","Data":"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"}
Sep 17 21:58:53.991063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.990971    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6" event={"ID":"099a8977-33e3-47e4-a09e-228b7920ade0","Type":"ContainerDied","Data":"193071ce1bd6e1dd6f67fcc759beb103dc4dc26e127e8e8650a0f0cd15b1d09d"}
Sep 17 21:58:53.991063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.991000    2587 scope.go:117] "RemoveContainer" containerID="ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"
Sep 17 21:58:53.992315 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.992263    2587 generic.go:356] "Generic (PLEG): container finished" podID="d2b475da-1067-429c-abdb-2294210bdf4d" containerID="d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175" exitCode=0
Sep 17 21:58:53.992419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.992323    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"
Sep 17 21:58:53.992419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.992338    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" event={"ID":"d2b475da-1067-429c-abdb-2294210bdf4d","Type":"ContainerDied","Data":"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"}
Sep 17 21:58:53.992419 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.992384    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k" event={"ID":"d2b475da-1067-429c-abdb-2294210bdf4d","Type":"ContainerDied","Data":"72ce7e626aa1aae31a94f39359d1f0ea1713884604d6891b338f7ecab026a160"}
Sep 17 21:58:53.999962 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:53.999946    2587 scope.go:117] "RemoveContainer" containerID="ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"
Sep 17 21:58:54.000241 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:58:54.000214    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b\": container with ID starting with ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b not found: ID does not exist" containerID="ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"
Sep 17 21:58:54.000358 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.000249    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b"} err="failed to get container status \"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b\": rpc error: code = NotFound desc = could not find container \"ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b\": container with ID starting with ab1bf80f6def90a67be1415b33dae9dd3cc03a406f9ab13c20751a66f006a64b not found: ID does not exist"
Sep 17 21:58:54.000358 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.000266    2587 scope.go:117] "RemoveContainer" containerID="d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"
Sep 17 21:58:54.008659 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.008642    2587 scope.go:117] "RemoveContainer" containerID="d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"
Sep 17 21:58:54.008954 ip-10-0-131-170 kubenswrapper[2587]: E0917 21:58:54.008935    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175\": container with ID starting with d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175 not found: ID does not exist" containerID="d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"
Sep 17 21:58:54.009015 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.008965    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175"} err="failed to get container status \"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175\": rpc error: code = NotFound desc = could not find container \"d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175\": container with ID starting with d1f8803fd995ddf52529849f5277407854b684aa3e8f5f2ec843dee4c7a89175 not found: ID does not exist"
Sep 17 21:58:54.035350 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035326    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume\") pod \"099a8977-33e3-47e4-a09e-228b7920ade0\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") "
Sep 17 21:58:54.035497 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035360    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume\") pod \"d2b475da-1067-429c-abdb-2294210bdf4d\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") "
Sep 17 21:58:54.035497 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035421    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gbfn\" (UniqueName: \"kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn\") pod \"d2b475da-1067-429c-abdb-2294210bdf4d\" (UID: \"d2b475da-1067-429c-abdb-2294210bdf4d\") "
Sep 17 21:58:54.035614 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035554    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8995\" (UniqueName: \"kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995\") pod \"099a8977-33e3-47e4-a09e-228b7920ade0\" (UID: \"099a8977-33e3-47e4-a09e-228b7920ade0\") "
Sep 17 21:58:54.035781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035760    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "099a8977-33e3-47e4-a09e-228b7920ade0" (UID: "099a8977-33e3-47e4-a09e-228b7920ade0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:54.035849 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035796    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2b475da-1067-429c-abdb-2294210bdf4d" (UID: "d2b475da-1067-429c-abdb-2294210bdf4d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:54.036241 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.035971    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099a8977-33e3-47e4-a09e-228b7920ade0-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:54.036241 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.036222    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b475da-1067-429c-abdb-2294210bdf4d-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:54.038122 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.038095    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995" (OuterVolumeSpecName: "kube-api-access-v8995") pod "099a8977-33e3-47e4-a09e-228b7920ade0" (UID: "099a8977-33e3-47e4-a09e-228b7920ade0"). InnerVolumeSpecName "kube-api-access-v8995". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:54.038488 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.038467    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn" (OuterVolumeSpecName: "kube-api-access-6gbfn") pod "d2b475da-1067-429c-abdb-2294210bdf4d" (UID: "d2b475da-1067-429c-abdb-2294210bdf4d"). InnerVolumeSpecName "kube-api-access-6gbfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:54.137160 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.137071    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8995\" (UniqueName: \"kubernetes.io/projected/099a8977-33e3-47e4-a09e-228b7920ade0-kube-api-access-v8995\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:54.137160 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.137098    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gbfn\" (UniqueName: \"kubernetes.io/projected/d2b475da-1067-429c-abdb-2294210bdf4d-kube-api-access-6gbfn\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 21:58:54.318660 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.318625    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:54.328539 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.328509    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c85107f9-6c77dd8f66-6qt8k"]
Sep 17 21:58:54.344210 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.344174    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:54.348726 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.348701    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c85107f9-66994bfdf7-8hsl6"]
Sep 17 21:58:54.857938 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.857903    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099a8977-33e3-47e4-a09e-228b7920ade0" path="/var/lib/kubelet/pods/099a8977-33e3-47e4-a09e-228b7920ade0/volumes"
Sep 17 21:58:54.858326 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:58:54.858315    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b475da-1067-429c-abdb-2294210bdf4d" path="/var/lib/kubelet/pods/d2b475da-1067-429c-abdb-2294210bdf4d/volumes"
Sep 17 21:59:08.435744 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.435706    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 21:59:08.436395 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436376    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b475da-1067-429c-abdb-2294210bdf4d" containerName="payload-pre-processing"
Sep 17 21:59:08.436449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436397    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b475da-1067-429c-abdb-2294210bdf4d" containerName="payload-pre-processing"
Sep 17 21:59:08.436449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436407    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099a8977-33e3-47e4-a09e-228b7920ade0" containerName="payload-processing"
Sep 17 21:59:08.436449 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436413    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="099a8977-33e3-47e4-a09e-228b7920ade0" containerName="payload-processing"
Sep 17 21:59:08.436545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436486    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="099a8977-33e3-47e4-a09e-228b7920ade0" containerName="payload-processing"
Sep 17 21:59:08.436545 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.436496    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b475da-1067-429c-abdb-2294210bdf4d" containerName="payload-pre-processing"
Sep 17 21:59:08.439565 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.439547    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.446394 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.446372    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f996ae\"/\"kube-root-ca.crt\""
Sep 17 21:59:08.446519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.446397    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f996ae\"/\"default-dockercfg-tqvp5\""
Sep 17 21:59:08.446519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.446445    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f996ae\"/\"premium-w0-f996ae-kserve-self-signed-certs\""
Sep 17 21:59:08.446519 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.446484    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f996ae\"/\"openshift-service-ca.crt\""
Sep 17 21:59:08.464871 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.464847    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 21:59:08.478899 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.478876    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4nmk\" (UniqueName: \"kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.478998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.478912    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.478998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.478937    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.478998 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.478973    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.479169 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.478998    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.479169 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.479106    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580577 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580542    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580733 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580588    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580733 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580685    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4nmk\" (UniqueName: \"kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580733 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580716    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580739    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.580881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.580761    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.581226 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.581194    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.581362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.581205    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.581362 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.581231    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.583079 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.583056    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.583211 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.583194    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.598471 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.598445    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4nmk\" (UniqueName: \"kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk\") pod \"premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") " pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:08.749162 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:08.749123    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:09.060099 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:09.060054    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 21:59:09.060334 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:59:09.060306    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce6cc4dc_3600_4717_9795_af4c6dddf674.slice/crio-355b78beedff959a0200665ed73fd287c86158dd86220b9a2945c034a58f7015 WatchSource:0}: Error finding container 355b78beedff959a0200665ed73fd287c86158dd86220b9a2945c034a58f7015: Status 404 returned error can't find the container with id 355b78beedff959a0200665ed73fd287c86158dd86220b9a2945c034a58f7015
Sep 17 21:59:09.078782 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:09.078756    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" event={"ID":"ce6cc4dc-3600-4717-9795-af4c6dddf674","Type":"ContainerStarted","Data":"355b78beedff959a0200665ed73fd287c86158dd86220b9a2945c034a58f7015"}
Sep 17 21:59:10.086000 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:10.085956    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" event={"ID":"ce6cc4dc-3600-4717-9795-af4c6dddf674","Type":"ContainerStarted","Data":"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"}
Sep 17 21:59:10.127596 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:10.127548    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" podStartSLOduration=2.127532249 podStartE2EDuration="2.127532249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:10.126581663 +0000 UTC m=+1053.475315542" watchObservedRunningTime="2026-09-17 21:59:10.127532249 +0000 UTC m=+1053.476266129"
Sep 17 21:59:18.750036 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:18.749941    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:18.750036 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:18.750021    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:18.763148 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:18.763124    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:19.144698 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:19.144622    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 21:59:38.382682 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.382648    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 21:59:38.386706 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.386685    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.391607 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.391582    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-dbe8a7\"/\"default-dockercfg-r9shz\""
Sep 17 21:59:38.391750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.391611    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-dbe8a7\"/\"openshift-service-ca.crt\""
Sep 17 21:59:38.391750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.391628    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-dbe8a7\"/\"premium-w4-dbe8a7-kserve-self-signed-certs\""
Sep 17 21:59:38.391750 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.391628    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-dbe8a7\"/\"kube-root-ca.crt\""
Sep 17 21:59:38.399513 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.399491    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 21:59:38.479187 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479153    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np6jj\" (UniqueName: \"kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.479187 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479188    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.479434 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479321    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.479434 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479355    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.479434 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479429    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.479538 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.479453    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580517    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580566 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580567    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580612    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580636    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580665    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np6jj\" (UniqueName: \"kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.580781 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.580688    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.581021 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.581000    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.581066 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.581019    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.581102 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.581068    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.583320 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.583268    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.583476 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.583457    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.590497 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.590480    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np6jj\" (UniqueName: \"kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj\") pod \"premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") " pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.700004 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.699922    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-dbe8a7\"/\"default-dockercfg-r9shz\""
Sep 17 21:59:38.707558 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.707535    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:38.985236 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:38.985212    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 21:59:38.987360 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:59:38.987331    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9be7797_ce44_4622_9013_03822f604568.slice/crio-60ec7ee9acdfa2ef8734ef53b53faf975ce3cde9d078ac011aa25ec8a69077e6 WatchSource:0}: Error finding container 60ec7ee9acdfa2ef8734ef53b53faf975ce3cde9d078ac011aa25ec8a69077e6: Status 404 returned error can't find the container with id 60ec7ee9acdfa2ef8734ef53b53faf975ce3cde9d078ac011aa25ec8a69077e6
Sep 17 21:59:39.241192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:39.241107    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" event={"ID":"e9be7797-ce44-4622-9013-03822f604568","Type":"ContainerStarted","Data":"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"}
Sep 17 21:59:39.241192 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:39.241153    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" event={"ID":"e9be7797-ce44-4622-9013-03822f604568","Type":"ContainerStarted","Data":"60ec7ee9acdfa2ef8734ef53b53faf975ce3cde9d078ac011aa25ec8a69077e6"}
Sep 17 21:59:39.266234 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:39.266189    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" podStartSLOduration=1.266174191 podStartE2EDuration="1.266174191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:39.265712521 +0000 UTC m=+1082.614446402" watchObservedRunningTime="2026-09-17 21:59:39.266174191 +0000 UTC m=+1082.614908094"
Sep 17 21:59:45.072063 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.072031    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 21:59:45.075691 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.075675    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.080021 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.079989    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3d5c73\"/\"kube-root-ca.crt\""
Sep 17 21:59:45.081475 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.081452    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3d5c73\"/\"test-model-3d5c73-kserve-self-signed-certs\""
Sep 17 21:59:45.081579 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.081522    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3d5c73\"/\"openshift-service-ca.crt\""
Sep 17 21:59:45.081579 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.081560    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3d5c73\"/\"default-dockercfg-x6mfr\""
Sep 17 21:59:45.101493 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.101467    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 21:59:45.150260 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150223    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.150472 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150309    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.150472 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150356    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4gs\" (UniqueName: \"kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.150583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150468    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.150583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150501    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.150583 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.150550    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251168 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251126    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251189    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251218    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4gs\" (UniqueName: \"kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251322    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251349    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251410 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251387    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251730 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251694    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251748    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.251854 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.251758    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.253741 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.253720    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.253881 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.253864    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.261695 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.261676    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4gs\" (UniqueName: \"kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs\") pod \"test-model-3d5c73-kserve-56d97bfc87-dptt4\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") " pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.386738 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.386647    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:45.683515 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:45.681083    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 21:59:45.684235 ip-10-0-131-170 kubenswrapper[2587]: W0917 21:59:45.684202    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca50fac7_10b2_44cc_8fb6_b4d666e76961.slice/crio-bc8856a4d323a6292069b388581b30475084c29a8ee24ecf1b9a2425fbb5fe68 WatchSource:0}: Error finding container bc8856a4d323a6292069b388581b30475084c29a8ee24ecf1b9a2425fbb5fe68: Status 404 returned error can't find the container with id bc8856a4d323a6292069b388581b30475084c29a8ee24ecf1b9a2425fbb5fe68
Sep 17 21:59:46.281386 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:46.281345    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" event={"ID":"ca50fac7-10b2-44cc-8fb6-b4d666e76961","Type":"ContainerStarted","Data":"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"}
Sep 17 21:59:46.281780 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:46.281399    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" event={"ID":"ca50fac7-10b2-44cc-8fb6-b4d666e76961","Type":"ContainerStarted","Data":"bc8856a4d323a6292069b388581b30475084c29a8ee24ecf1b9a2425fbb5fe68"}
Sep 17 21:59:46.314413 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:46.314363    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" podStartSLOduration=1.314347388 podStartE2EDuration="1.314347388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:46.312050424 +0000 UTC m=+1089.660784307" watchObservedRunningTime="2026-09-17 21:59:46.314347388 +0000 UTC m=+1089.663081267"
Sep 17 21:59:48.708026 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:48.707992    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:48.708446 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:48.708051    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:48.720515 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:48.720489    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:49.313533 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:49.313491    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 21:59:55.387861 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:55.387807    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:55.387861 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:55.387876    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:55.400174 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:55.400150    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 21:59:56.346735 ip-10-0-131-170 kubenswrapper[2587]: I0917 21:59:56.346705    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 22:00:00.201507 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.201473    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"]
Sep 17 22:00:00.208367 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.206996    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"
Sep 17 22:00:00.211692 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.211647    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmdbp\""
Sep 17 22:00:00.220572 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.220550    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"
Sep 17 22:00:00.226964 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.226932    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"]
Sep 17 22:00:00.504672 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.504634    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"]
Sep 17 22:00:00.510363 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:00.510342    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 22:00:01.363703 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:01.363667    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s" event={"ID":"b29d3041-2f84-4416-b571-93d180098449","Type":"ContainerStarted","Data":"af2ed7163181dd404b9522c23ea4eab16dcb622cc4bf0c4604690838c0342a41"}
Sep 17 22:00:04.385324 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:04.385277    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s" event={"ID":"b29d3041-2f84-4416-b571-93d180098449","Type":"ContainerStarted","Data":"db99cb2a4980e3406a65d4822cc2ff7b28bd0b3d2e0bf32e528c8ab43569c445"}
Sep 17 22:00:04.409926 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:04.409868    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s" podStartSLOduration=1.255155445 podStartE2EDuration="4.409850815s" totalImagesPullingTime="3.15469537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:00:04.408149974 +0000 UTC m=+1107.756883854" watchObservedRunningTime="2026-09-17 22:00:04.409850815 +0000 UTC m=+1107.758584694"
Sep 17 22:00:05.402133 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:05.402094    2587 generic.go:356] "Generic (PLEG): container finished" podID="b29d3041-2f84-4416-b571-93d180098449" containerID="db99cb2a4980e3406a65d4822cc2ff7b28bd0b3d2e0bf32e528c8ab43569c445" exitCode=0
Sep 17 22:00:05.402581 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:05.402182    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s" event={"ID":"b29d3041-2f84-4416-b571-93d180098449","Type":"ContainerDied","Data":"db99cb2a4980e3406a65d4822cc2ff7b28bd0b3d2e0bf32e528c8ab43569c445"}
Sep 17 22:00:06.550554 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:06.550529    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"
Sep 17 22:00:07.415133 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:07.415103    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"
Sep 17 22:00:07.415303 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:07.415139    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s" event={"ID":"b29d3041-2f84-4416-b571-93d180098449","Type":"ContainerDied","Data":"af2ed7163181dd404b9522c23ea4eab16dcb622cc4bf0c4604690838c0342a41"}
Sep 17 22:00:07.415303 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:07.415185    2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af2ed7163181dd404b9522c23ea4eab16dcb622cc4bf0c4604690838c0342a41"
Sep 17 22:00:42.079738 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:42.079698    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"]
Sep 17 22:00:42.084811 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:42.084781    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-4bd0ef-29828040-9454s"]
Sep 17 22:00:42.857575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:42.857539    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b29d3041-2f84-4416-b571-93d180098449" path="/var/lib/kubelet/pods/b29d3041-2f84-4416-b571-93d180098449/volumes"
Sep 17 22:00:50.270629 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.270537    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:00:50.271077 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.271031    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b29d3041-2f84-4416-b571-93d180098449" containerName="cleanup"
Sep 17 22:00:50.271077 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.271045    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b29d3041-2f84-4416-b571-93d180098449" containerName="cleanup"
Sep 17 22:00:50.271152 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.271142    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b29d3041-2f84-4416-b571-93d180098449" containerName="cleanup"
Sep 17 22:00:50.274429 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.274408    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.279032 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.279008    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-93af11bc-dockercfg-pllft\""
Sep 17 22:00:50.280571 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.280552    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-93af11bc\""
Sep 17 22:00:50.291620 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.291592    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:00:50.328617 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.328583    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ltvw\" (UniqueName: \"kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.328617 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.328633    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.430080 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.430043    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ltvw\" (UniqueName: \"kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.430250 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.430094    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.430928 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.430903    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.450138 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.450115    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ltvw\" (UniqueName: \"kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw\") pod \"payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.585220 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.585130    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:50.864775 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:50.864748    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:00:50.866372 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:00:50.866347    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bec93e_0557_496c_8e04_94ed07d9a912.slice/crio-59c1777afbb7ab23308115ec70b7e284868879f600c452d2c5347bfdb1dc6897 WatchSource:0}: Error finding container 59c1777afbb7ab23308115ec70b7e284868879f600c452d2c5347bfdb1dc6897: Status 404 returned error can't find the container with id 59c1777afbb7ab23308115ec70b7e284868879f600c452d2c5347bfdb1dc6897
Sep 17 22:00:51.667796 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:51.667752    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" event={"ID":"e2bec93e-0557-496c-8e04-94ed07d9a912","Type":"ContainerStarted","Data":"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"}
Sep 17 22:00:51.668265 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:51.667846    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" event={"ID":"e2bec93e-0557-496c-8e04-94ed07d9a912","Type":"ContainerStarted","Data":"59c1777afbb7ab23308115ec70b7e284868879f600c452d2c5347bfdb1dc6897"}
Sep 17 22:00:51.668265 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:51.667888    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:00:51.692211 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:00:51.692160    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" podStartSLOduration=1.692144756 podStartE2EDuration="1.692144756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:00:51.688553317 +0000 UTC m=+1155.037287204" watchObservedRunningTime="2026-09-17 22:00:51.692144756 +0000 UTC m=+1155.040878664"
Sep 17 22:01:02.675785 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:02.675754    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:01:34.066910 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.066871    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:01:34.067505 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.067155    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" podUID="e2bec93e-0557-496c-8e04-94ed07d9a912" containerName="payload-pre-processing" containerID="cri-o://ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e" gracePeriod=30
Sep 17 22:01:34.338952 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.338907    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:01:34.496733 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.496681    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ltvw\" (UniqueName: \"kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw\") pod \"e2bec93e-0557-496c-8e04-94ed07d9a912\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") "
Sep 17 22:01:34.496930 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.496791    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume\") pod \"e2bec93e-0557-496c-8e04-94ed07d9a912\" (UID: \"e2bec93e-0557-496c-8e04-94ed07d9a912\") "
Sep 17 22:01:34.497206 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.497180    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2bec93e-0557-496c-8e04-94ed07d9a912" (UID: "e2bec93e-0557-496c-8e04-94ed07d9a912"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:34.499197 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.499168    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw" (OuterVolumeSpecName: "kube-api-access-5ltvw") pod "e2bec93e-0557-496c-8e04-94ed07d9a912" (UID: "e2bec93e-0557-496c-8e04-94ed07d9a912"). InnerVolumeSpecName "kube-api-access-5ltvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:01:34.598405 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.598332    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2bec93e-0557-496c-8e04-94ed07d9a912-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:01:34.598405 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.598358    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ltvw\" (UniqueName: \"kubernetes.io/projected/e2bec93e-0557-496c-8e04-94ed07d9a912-kube-api-access-5ltvw\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:01:34.910881 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.910848    2587 generic.go:356] "Generic (PLEG): container finished" podID="e2bec93e-0557-496c-8e04-94ed07d9a912" containerID="ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e" exitCode=0
Sep 17 22:01:34.911030 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.910904    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"
Sep 17 22:01:34.911030 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.910937    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" event={"ID":"e2bec93e-0557-496c-8e04-94ed07d9a912","Type":"ContainerDied","Data":"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"}
Sep 17 22:01:34.911030 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.910990    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx" event={"ID":"e2bec93e-0557-496c-8e04-94ed07d9a912","Type":"ContainerDied","Data":"59c1777afbb7ab23308115ec70b7e284868879f600c452d2c5347bfdb1dc6897"}
Sep 17 22:01:34.911030 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.911011    2587 scope.go:117] "RemoveContainer" containerID="ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"
Sep 17 22:01:34.920080 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.920058    2587 scope.go:117] "RemoveContainer" containerID="ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"
Sep 17 22:01:34.920360 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:01:34.920340    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e\": container with ID starting with ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e not found: ID does not exist" containerID="ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"
Sep 17 22:01:34.920419 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.920368    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e"} err="failed to get container status \"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e\": rpc error: code = NotFound desc = could not find container \"ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e\": container with ID starting with ae29e88e25b7830d8b69ec33fad8f3ea58808133409dcd33720baa340e4e391e not found: ID does not exist"
Sep 17 22:01:34.949571 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.949542    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:01:34.956515 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:34.956492    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-93af11bc-6699cf7c9d-sbjnx"]
Sep 17 22:01:36.859057 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:36.859016    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2bec93e-0557-496c-8e04-94ed07d9a912" path="/var/lib/kubelet/pods/e2bec93e-0557-496c-8e04-94ed07d9a912/volumes"
Sep 17 22:01:38.848363 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:38.848339    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:01:38.848800 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:01:38.848374    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:02:05.628859 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.628817    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:02:05.629366 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.629260    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2bec93e-0557-496c-8e04-94ed07d9a912" containerName="payload-pre-processing"
Sep 17 22:02:05.629366 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.629271    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2bec93e-0557-496c-8e04-94ed07d9a912" containerName="payload-pre-processing"
Sep 17 22:02:05.629451 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.629369    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2bec93e-0557-496c-8e04-94ed07d9a912" containerName="payload-pre-processing"
Sep 17 22:02:05.631790 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.631773    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.635018 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.634991    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a23afc\""
Sep 17 22:02:05.635145 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.635087    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a23afc-dockercfg-x7qlf\""
Sep 17 22:02:05.647683 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.647658    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:02:05.719298 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.719257    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.719499 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.719406    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4t7\" (UniqueName: \"kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.820737 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.820695    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.820909 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.820782    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4t7\" (UniqueName: \"kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.821469 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.821449    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.830619 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.830593    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4t7\" (UniqueName: \"kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7\") pod \"payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:05.941987 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:05.941902    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:06.220451 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:06.220424    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:02:06.222146 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:02:06.222122    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ee5a3c_c72a_4231_8802_4cd66f8804b4.slice/crio-c7ad2e58618d1f6c59c3ecad93d144945a3108f2476130a537bb2d4bab1e818c WatchSource:0}: Error finding container c7ad2e58618d1f6c59c3ecad93d144945a3108f2476130a537bb2d4bab1e818c: Status 404 returned error can't find the container with id c7ad2e58618d1f6c59c3ecad93d144945a3108f2476130a537bb2d4bab1e818c
Sep 17 22:02:07.090530 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:07.090454    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" event={"ID":"39ee5a3c-c72a-4231-8802-4cd66f8804b4","Type":"ContainerStarted","Data":"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"}
Sep 17 22:02:07.090998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:07.090556    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:07.090998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:07.090579    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" event={"ID":"39ee5a3c-c72a-4231-8802-4cd66f8804b4","Type":"ContainerStarted","Data":"c7ad2e58618d1f6c59c3ecad93d144945a3108f2476130a537bb2d4bab1e818c"}
Sep 17 22:02:07.116776 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:07.116716    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" podStartSLOduration=2.116699144 podStartE2EDuration="2.116699144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:07.114690124 +0000 UTC m=+1230.463424006" watchObservedRunningTime="2026-09-17 22:02:07.116699144 +0000 UTC m=+1230.465433073"
Sep 17 22:02:18.097915 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:18.097882    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:02:54.894866 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:54.894832    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7820-a8a169"]
Sep 17 22:02:54.897605 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:54.897581    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:02:54.904264 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:54.904238    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lbp7g\""
Sep 17 22:02:54.911878 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:54.911855    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7820-a8a169"]
Sep 17 22:02:55.008771 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:55.008728    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cn9n\" (UniqueName: \"kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n\") pod \"test-curl-7820-a8a169\" (UID: \"991e243d-8eea-4fa6-a114-bf80ac4c5a2c\") " pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:02:55.110236 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:55.110194    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cn9n\" (UniqueName: \"kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n\") pod \"test-curl-7820-a8a169\" (UID: \"991e243d-8eea-4fa6-a114-bf80ac4c5a2c\") " pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:02:55.126834 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:55.126803    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cn9n\" (UniqueName: \"kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n\") pod \"test-curl-7820-a8a169\" (UID: \"991e243d-8eea-4fa6-a114-bf80ac4c5a2c\") " pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:02:55.207078 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:55.206962    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:02:55.513149 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:55.513123    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7820-a8a169"]
Sep 17 22:02:55.514000 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:02:55.513974    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod991e243d_8eea_4fa6_a114_bf80ac4c5a2c.slice/crio-bd8f08cec73e339114154a9925d46d0d8ad74f21b8bab867aa48b0b23531a598 WatchSource:0}: Error finding container bd8f08cec73e339114154a9925d46d0d8ad74f21b8bab867aa48b0b23531a598: Status 404 returned error can't find the container with id bd8f08cec73e339114154a9925d46d0d8ad74f21b8bab867aa48b0b23531a598
Sep 17 22:02:56.363252 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:56.363215    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7820-a8a169" event={"ID":"991e243d-8eea-4fa6-a114-bf80ac4c5a2c","Type":"ContainerStarted","Data":"bd8f08cec73e339114154a9925d46d0d8ad74f21b8bab867aa48b0b23531a598"}
Sep 17 22:02:58.377688 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:58.377646    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7820-a8a169" event={"ID":"991e243d-8eea-4fa6-a114-bf80ac4c5a2c","Type":"ContainerStarted","Data":"c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e"}
Sep 17 22:02:59.065335 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:59.065261    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7820-a8a169" podStartSLOduration=2.902615008 podStartE2EDuration="5.065242378s" totalImagesPullingTime="2.16262737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:58.400826025 +0000 UTC m=+1281.749559905" watchObservedRunningTime="2026-09-17 22:02:59.065242378 +0000 UTC m=+1282.413976257"
Sep 17 22:02:59.065512 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:59.065409    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7820-a8a169"]
Sep 17 22:02:59.070632 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:02:59.070605    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7820-a8a169"]
Sep 17 22:02:59.382849 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:02:59.382820    2587 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7820-a8a169" container="test-curl-7820-a8a169"
Sep 17 22:03:00.388320 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:00.388248    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7820-a8a169" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" containerName="test-curl-7820-a8a169" containerID="cri-o://c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e" gracePeriod=2
Sep 17 22:03:00.391103 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:03:00.391069    2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7820-a8a169\" is forbidden: User \"system:node:ip-10-0-131-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-170.ec2.internal' and this object" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:03:02.586206 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:02.586176    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:03:02.690913 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:02.690831    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cn9n\" (UniqueName: \"kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n\") pod \"991e243d-8eea-4fa6-a114-bf80ac4c5a2c\" (UID: \"991e243d-8eea-4fa6-a114-bf80ac4c5a2c\") "
Sep 17 22:03:02.693128 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:02.693102    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n" (OuterVolumeSpecName: "kube-api-access-4cn9n") pod "991e243d-8eea-4fa6-a114-bf80ac4c5a2c" (UID: "991e243d-8eea-4fa6-a114-bf80ac4c5a2c"). InnerVolumeSpecName "kube-api-access-4cn9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:03:02.792692 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:02.792657    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cn9n\" (UniqueName: \"kubernetes.io/projected/991e243d-8eea-4fa6-a114-bf80ac4c5a2c-kube-api-access-4cn9n\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:02.857513 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:02.857479    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" path="/var/lib/kubelet/pods/991e243d-8eea-4fa6-a114-bf80ac4c5a2c/volumes"
Sep 17 22:03:03.405440 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:03.405395    2587 generic.go:356] "Generic (PLEG): container finished" podID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" containerID="c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e" exitCode=137
Sep 17 22:03:03.405624 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:03.405447    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7820-a8a169"
Sep 17 22:03:03.405624 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:03.405491    2587 scope.go:117] "RemoveContainer" containerID="c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e"
Sep 17 22:03:03.414996 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:03.414969    2587 scope.go:117] "RemoveContainer" containerID="c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e"
Sep 17 22:03:03.415305 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:03:03.415263    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e\": container with ID starting with c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e not found: ID does not exist" containerID="c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e"
Sep 17 22:03:03.415389 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:03.415318    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e"} err="failed to get container status \"c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e\": rpc error: code = NotFound desc = could not find container \"c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e\": container with ID starting with c69eefe4d92997094fd7f725da20e82a2fb63b9a00ebf410c067d722fe27177e not found: ID does not exist"
Sep 17 22:03:17.075133 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.075090    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:03:17.075714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.075687    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" containerName="test-curl-7820-a8a169"
Sep 17 22:03:17.075714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.075709    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" containerName="test-curl-7820-a8a169"
Sep 17 22:03:17.075908 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.075818    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="991e243d-8eea-4fa6-a114-bf80ac4c5a2c" containerName="test-curl-7820-a8a169"
Sep 17 22:03:17.078827 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.078807    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.082259 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.082239    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d8ea1ea7-serving-cert\""
Sep 17 22:03:17.128064 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.128035    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:03:17.245138 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.245106    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.245403 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.245159    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-887qm\" (UniqueName: \"kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.245403 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.245342    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.245403 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.245393    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.346027 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.345937    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.346027 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.345982    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.346027 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.346022    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.346257 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.346052    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-887qm\" (UniqueName: \"kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.346716 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.346696    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.348603 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.348575    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.348660 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.348646    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.362371 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.362342    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-887qm\" (UniqueName: \"kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm\") pod \"maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.389299 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.389260    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:17.712092 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:17.712064    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:03:17.713975 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:03:17.713933    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f3badd6_8f9b_4d56_9142_b041bc8dca2e.slice/crio-a1deba5a3729b6bd545f758f77f5145c191c8225703c98f900563b4ca648a1d5 WatchSource:0}: Error finding container a1deba5a3729b6bd545f758f77f5145c191c8225703c98f900563b4ca648a1d5: Status 404 returned error can't find the container with id a1deba5a3729b6bd545f758f77f5145c191c8225703c98f900563b4ca648a1d5
Sep 17 22:03:18.492968 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:18.492876    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" event={"ID":"6f3badd6-8f9b-4d56-9142-b041bc8dca2e","Type":"ContainerStarted","Data":"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"}
Sep 17 22:03:18.492968 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:18.492931    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" event={"ID":"6f3badd6-8f9b-4d56-9142-b041bc8dca2e","Type":"ContainerStarted","Data":"a1deba5a3729b6bd545f758f77f5145c191c8225703c98f900563b4ca648a1d5"}
Sep 17 22:03:18.493448 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:18.492991    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:18.516138 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:18.516090    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" podStartSLOduration=0.990119972 podStartE2EDuration="1.516068285s" totalImagesPullingTime="525.948313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:03:18.515199252 +0000 UTC m=+1301.863933133" watchObservedRunningTime="2026-09-17 22:03:18.516068285 +0000 UTC m=+1301.864802168"
Sep 17 22:03:21.608435 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.608397    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 22:03:21.608912 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.608639    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" podUID="f36bef8f-a2bd-4228-897b-a3bcdda89431" containerName="payload-pre-processing" containerID="cri-o://23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d" gracePeriod=30
Sep 17 22:03:21.872915 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.872856    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 22:03:21.893079 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.893051    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrc7m\" (UniqueName: \"kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m\") pod \"f36bef8f-a2bd-4228-897b-a3bcdda89431\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") "
Sep 17 22:03:21.893226 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.893132    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume\") pod \"f36bef8f-a2bd-4228-897b-a3bcdda89431\" (UID: \"f36bef8f-a2bd-4228-897b-a3bcdda89431\") "
Sep 17 22:03:21.893652 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.893474    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f36bef8f-a2bd-4228-897b-a3bcdda89431" (UID: "f36bef8f-a2bd-4228-897b-a3bcdda89431"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:03:21.893652 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.893617    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36bef8f-a2bd-4228-897b-a3bcdda89431-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:21.895242 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.895217    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m" (OuterVolumeSpecName: "kube-api-access-wrc7m") pod "f36bef8f-a2bd-4228-897b-a3bcdda89431" (UID: "f36bef8f-a2bd-4228-897b-a3bcdda89431"). InnerVolumeSpecName "kube-api-access-wrc7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:03:21.994656 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:21.994618    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrc7m\" (UniqueName: \"kubernetes.io/projected/f36bef8f-a2bd-4228-897b-a3bcdda89431-kube-api-access-wrc7m\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:22.517449 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.517412    2587 generic.go:356] "Generic (PLEG): container finished" podID="f36bef8f-a2bd-4228-897b-a3bcdda89431" containerID="23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d" exitCode=0
Sep 17 22:03:22.517649 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.517454    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"
Sep 17 22:03:22.517649 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.517500    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" event={"ID":"f36bef8f-a2bd-4228-897b-a3bcdda89431","Type":"ContainerDied","Data":"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"}
Sep 17 22:03:22.517649 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.517558    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7" event={"ID":"f36bef8f-a2bd-4228-897b-a3bcdda89431","Type":"ContainerDied","Data":"b60146de0f1685a7a711fc12e96f940b7f5a1b308c598acd480ff2608cc9474b"}
Sep 17 22:03:22.517649 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.517586    2587 scope.go:117] "RemoveContainer" containerID="23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"
Sep 17 22:03:22.527320 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.527267    2587 scope.go:117] "RemoveContainer" containerID="23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"
Sep 17 22:03:22.527590 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:03:22.527571    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d\": container with ID starting with 23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d not found: ID does not exist" containerID="23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"
Sep 17 22:03:22.527644 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.527600    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d"} err="failed to get container status \"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d\": rpc error: code = NotFound desc = could not find container \"23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d\": container with ID starting with 23c5cfb5b626a1e27f4152903d7f887f600c344b7809a96e8e9bb2462914d25d not found: ID does not exist"
Sep 17 22:03:22.581687 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.581660    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 22:03:22.610752 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.610725    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d5c73-5fcc759cdb-hwnj7"]
Sep 17 22:03:22.857608 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:22.857532    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36bef8f-a2bd-4228-897b-a3bcdda89431" path="/var/lib/kubelet/pods/f36bef8f-a2bd-4228-897b-a3bcdda89431/volumes"
Sep 17 22:03:24.505018 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:24.504984    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:03:27.307393 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:27.307354    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 22:03:42.327971 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.327920    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" podUID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" containerName="main" containerID="cri-o://432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876" gracePeriod=45
Sep 17 22:03:42.584640 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.584585    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 22:03:42.641152 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.641114    2587 generic.go:356] "Generic (PLEG): container finished" podID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" containerID="432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876" exitCode=0
Sep 17 22:03:42.641324 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.641168    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"
Sep 17 22:03:42.641324 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.641199    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" event={"ID":"ca50fac7-10b2-44cc-8fb6-b4d666e76961","Type":"ContainerDied","Data":"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"}
Sep 17 22:03:42.641324 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.641256    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4" event={"ID":"ca50fac7-10b2-44cc-8fb6-b4d666e76961","Type":"ContainerDied","Data":"bc8856a4d323a6292069b388581b30475084c29a8ee24ecf1b9a2425fbb5fe68"}
Sep 17 22:03:42.641324 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.641270    2587 scope.go:117] "RemoveContainer" containerID="432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"
Sep 17 22:03:42.650584 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.650566    2587 scope.go:117] "RemoveContainer" containerID="432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"
Sep 17 22:03:42.650844 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:03:42.650825    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876\": container with ID starting with 432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876 not found: ID does not exist" containerID="432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"
Sep 17 22:03:42.650897 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.650852    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876"} err="failed to get container status \"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876\": rpc error: code = NotFound desc = could not find container \"432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876\": container with ID starting with 432c6ce408088df26be23ab5e3f49bab54f951f11db5be0038ae348ff39e8876 not found: ID does not exist"
Sep 17 22:03:42.692435 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692396    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692445    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692500    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692527    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692592    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692808 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692624    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h4gs\" (UniqueName: \"kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs\") pod \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\" (UID: \"ca50fac7-10b2-44cc-8fb6-b4d666e76961\") "
Sep 17 22:03:42.692808 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692712    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:03:42.692808 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692791    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:03:42.692939 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692828    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home" (OuterVolumeSpecName: "home") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:03:42.693009 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.692986    2587 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tmp-dir\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.693158 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.693013    2587 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-model-cache\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.693158 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.693025    2587 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-home\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.695027 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.694996    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs" (OuterVolumeSpecName: "kube-api-access-9h4gs") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "kube-api-access-9h4gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:03:42.695027 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.695015    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm" (OuterVolumeSpecName: "dshm") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:03:42.695157 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.695067    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca50fac7-10b2-44cc-8fb6-b4d666e76961" (UID: "ca50fac7-10b2-44cc-8fb6-b4d666e76961"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:03:42.794472 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.794437    2587 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca50fac7-10b2-44cc-8fb6-b4d666e76961-tls-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.794472 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.794467    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h4gs\" (UniqueName: \"kubernetes.io/projected/ca50fac7-10b2-44cc-8fb6-b4d666e76961-kube-api-access-9h4gs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.794472 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.794476    2587 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca50fac7-10b2-44cc-8fb6-b4d666e76961-dshm\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:03:42.959780 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.959723    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 22:03:42.962421 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:42.962395    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3d5c73/test-model-3d5c73-kserve-56d97bfc87-dptt4"]
Sep 17 22:03:44.858026 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:03:44.857990    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" path="/var/lib/kubelet/pods/ca50fac7-10b2-44cc-8fb6-b4d666e76961/volumes"
Sep 17 22:04:14.622130 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622051    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:04:14.622616 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622599    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" containerName="main"
Sep 17 22:04:14.622616 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622615    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" containerName="main"
Sep 17 22:04:14.622699 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622657    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36bef8f-a2bd-4228-897b-a3bcdda89431" containerName="payload-pre-processing"
Sep 17 22:04:14.622699 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622663    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36bef8f-a2bd-4228-897b-a3bcdda89431" containerName="payload-pre-processing"
Sep 17 22:04:14.622765 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622728    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca50fac7-10b2-44cc-8fb6-b4d666e76961" containerName="main"
Sep 17 22:04:14.622765 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.622740    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36bef8f-a2bd-4228-897b-a3bcdda89431" containerName="payload-pre-processing"
Sep 17 22:04:14.625159 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.625143    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.628433 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.628410    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-8f8dad\""
Sep 17 22:04:14.643738 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.643707    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:04:14.706947 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.706909    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.706947 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.706943    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qj8b\" (UniqueName: \"kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.808476 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.808430    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.808476 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.808470    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qj8b\" (UniqueName: \"kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.809173 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.809149    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.818535 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.818509    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qj8b\" (UniqueName: \"kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b\") pod \"payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:14.936548 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:14.936464    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:15.230767 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:15.230743    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:04:15.232634 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:04:15.232605    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb698c377_c199_4fe3_b28a_4ce61424631f.slice/crio-bd410d65ca1948d78af018de4b5249247f66d649b077f5572cec7f6c168cc184 WatchSource:0}: Error finding container bd410d65ca1948d78af018de4b5249247f66d649b077f5572cec7f6c168cc184: Status 404 returned error can't find the container with id bd410d65ca1948d78af018de4b5249247f66d649b077f5572cec7f6c168cc184
Sep 17 22:04:15.828354 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:15.828311    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" event={"ID":"b698c377-c199-4fe3-b28a-4ce61424631f","Type":"ContainerStarted","Data":"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"}
Sep 17 22:04:15.828741 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:15.828470    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:15.828741 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:15.828494    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" event={"ID":"b698c377-c199-4fe3-b28a-4ce61424631f","Type":"ContainerStarted","Data":"bd410d65ca1948d78af018de4b5249247f66d649b077f5572cec7f6c168cc184"}
Sep 17 22:04:15.856492 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:15.856431    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" podStartSLOduration=1.8564127799999999 podStartE2EDuration="1.85641278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:04:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:04:15.852338936 +0000 UTC m=+1359.201072816" watchObservedRunningTime="2026-09-17 22:04:15.85641278 +0000 UTC m=+1359.205146665"
Sep 17 22:04:26.840072 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:26.840037    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:04:50.379552 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.379513    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:04:50.385672 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.385645    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.389004 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.388977    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-35b181-dockercfg-7r6tc\""
Sep 17 22:04:50.389382 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.389360    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-35b181\""
Sep 17 22:04:50.389773 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.389745    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:04:50.478383 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.478343    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf8h9\" (UniqueName: \"kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.478563 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.478407    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.579346 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.579306    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf8h9\" (UniqueName: \"kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.579531 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.579361    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.580067 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.580047    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.591388 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.591355    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf8h9\" (UniqueName: \"kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9\") pod \"payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.697972 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.697887    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:50.995801 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:04:50.995771    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d172c6_b413_4183_a516_04d710e2a537.slice/crio-c3f09360dc91c8d19e0d07324175fafcfbcd3770e49c16bb4ad79b9a5c6e2671 WatchSource:0}: Error finding container c3f09360dc91c8d19e0d07324175fafcfbcd3770e49c16bb4ad79b9a5c6e2671: Status 404 returned error can't find the container with id c3f09360dc91c8d19e0d07324175fafcfbcd3770e49c16bb4ad79b9a5c6e2671
Sep 17 22:04:50.997325 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:50.997267    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:04:51.033329 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:51.033278    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" event={"ID":"b2d172c6-b413-4183-a516-04d710e2a537","Type":"ContainerStarted","Data":"c3f09360dc91c8d19e0d07324175fafcfbcd3770e49c16bb4ad79b9a5c6e2671"}
Sep 17 22:04:52.040252 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:52.040219    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" event={"ID":"b2d172c6-b413-4183-a516-04d710e2a537","Type":"ContainerStarted","Data":"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"}
Sep 17 22:04:52.040704 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:52.040356    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:04:52.062852 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:04:52.062804    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" podStartSLOduration=2.062787395 podStartE2EDuration="2.062787395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:04:52.059189434 +0000 UTC m=+1395.407923320" watchObservedRunningTime="2026-09-17 22:04:52.062787395 +0000 UTC m=+1395.411521275"
Sep 17 22:05:03.048457 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:03.048425    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:05:38.922902 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:38.922872    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:05:38.923482 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:38.923112    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" podUID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" containerName="maas-api" containerID="cri-o://4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae" gracePeriod=30
Sep 17 22:05:39.182446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.182384    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:05:39.266264 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266233    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls\") pod \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") "
Sep 17 22:05:39.266464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266278    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca\") pod \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") "
Sep 17 22:05:39.266464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266449    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs\") pod \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") "
Sep 17 22:05:39.266562 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266513    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-887qm\" (UniqueName: \"kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm\") pod \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\" (UID: \"6f3badd6-8f9b-4d56-9142-b041bc8dca2e\") "
Sep 17 22:05:39.266735 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266691    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f3badd6-8f9b-4d56-9142-b041bc8dca2e" (UID: "6f3badd6-8f9b-4d56-9142-b041bc8dca2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:05:39.266973 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.266949    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:05:39.268734 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.268712    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f3badd6-8f9b-4d56-9142-b041bc8dca2e" (UID: "6f3badd6-8f9b-4d56-9142-b041bc8dca2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:05:39.268834 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.268819    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f3badd6-8f9b-4d56-9142-b041bc8dca2e" (UID: "6f3badd6-8f9b-4d56-9142-b041bc8dca2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:05:39.269165 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.269139    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm" (OuterVolumeSpecName: "kube-api-access-887qm") pod "6f3badd6-8f9b-4d56-9142-b041bc8dca2e" (UID: "6f3badd6-8f9b-4d56-9142-b041bc8dca2e"). InnerVolumeSpecName "kube-api-access-887qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:05:39.304000 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.303962    2587 generic.go:356] "Generic (PLEG): container finished" podID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" containerID="4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae" exitCode=0
Sep 17 22:05:39.304137 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.304003    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"
Sep 17 22:05:39.304137 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.304046    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" event={"ID":"6f3badd6-8f9b-4d56-9142-b041bc8dca2e","Type":"ContainerDied","Data":"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"}
Sep 17 22:05:39.304137 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.304100    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9" event={"ID":"6f3badd6-8f9b-4d56-9142-b041bc8dca2e","Type":"ContainerDied","Data":"a1deba5a3729b6bd545f758f77f5145c191c8225703c98f900563b4ca648a1d5"}
Sep 17 22:05:39.304137 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.304125    2587 scope.go:117] "RemoveContainer" containerID="4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"
Sep 17 22:05:39.317131 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.317112    2587 scope.go:117] "RemoveContainer" containerID="4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"
Sep 17 22:05:39.317473 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:05:39.317455    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae\": container with ID starting with 4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae not found: ID does not exist" containerID="4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"
Sep 17 22:05:39.317525 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.317482    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae"} err="failed to get container status \"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae\": rpc error: code = NotFound desc = could not find container \"4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae\": container with ID starting with 4b72abbe23201514046b6e2d2696ce22496bb6822099849ec610efe80548e5ae not found: ID does not exist"
Sep 17 22:05:39.334723 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.334690    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:05:39.337480 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.337456    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d8ea1ea7-5c845fcfcc-2gvw9"]
Sep 17 22:05:39.368029 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.367995    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:05:39.368029 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.368024    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:05:39.368029 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:39.368033    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-887qm\" (UniqueName: \"kubernetes.io/projected/6f3badd6-8f9b-4d56-9142-b041bc8dca2e-kube-api-access-887qm\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:05:40.857352 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:05:40.857310    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" path="/var/lib/kubelet/pods/6f3badd6-8f9b-4d56-9142-b041bc8dca2e/volumes"
Sep 17 22:06:38.885236 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:06:38.885211    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:06:38.888013 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:06:38.887995    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:06:38.974826 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:06:38.974794    2587 scope.go:117] "RemoveContainer" containerID="db99cb2a4980e3406a65d4822cc2ff7b28bd0b3d2e0bf32e528c8ab43569c445"
Sep 17 22:08:00.531472 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.531440    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:08:00.533714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.532133    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" containerName="maas-api"
Sep 17 22:08:00.533714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.532150    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" containerName="maas-api"
Sep 17 22:08:00.533714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.532216    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f3badd6-8f9b-4d56-9142-b041bc8dca2e" containerName="maas-api"
Sep 17 22:08:00.535443 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.535396    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.538509 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.538478    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-194000-serving-cert\""
Sep 17 22:08:00.550194 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.550166    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:08:00.687890 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.687851    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxftj\" (UniqueName: \"kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.688058 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.687903    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.688058 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.687969    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.688138 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.688057    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.788698 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.788602    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxftj\" (UniqueName: \"kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.788698 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.788673    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.788953 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.788727    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.788953 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.788776    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.788953 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:08:00.788824    2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-194000-serving-cert: secret "maas-api-e2e-api-a-194000-serving-cert" not found
Sep 17 22:08:00.788953 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:08:00.788893    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls podName:cf8b7146-727c-4bb7-a9ae-e707cb3856af nodeName:}" failed. No retries permitted until 2026-09-17 22:08:01.288874001 +0000 UTC m=+1584.637607863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls") pod "maas-api-e2e-api-a-194000-86f845648b-9w46k" (UID: "cf8b7146-727c-4bb7-a9ae-e707cb3856af") : secret "maas-api-e2e-api-a-194000-serving-cert" not found
Sep 17 22:08:00.789588 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.789565    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.791474 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.791454    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:00.804201 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:00.804163    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxftj\" (UniqueName: \"kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:01.294990 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.294957    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:01.297511 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.297490    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") pod \"maas-api-e2e-api-a-194000-86f845648b-9w46k\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:01.401354 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.401318    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:08:01.405319 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.405270    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.408768 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.408743    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-194000\""
Sep 17 22:08:01.408875 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.408788    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-194000-dockercfg-b7rjd\""
Sep 17 22:08:01.418411 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.418379    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:08:01.447327 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.447245    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:01.497436 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.497404    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.497615 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.497471    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftrs8\" (UniqueName: \"kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.598541 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.598441    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.598541 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.598497    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftrs8\" (UniqueName: \"kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.599248 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.599227    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.608039 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.608009    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftrs8\" (UniqueName: \"kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8\") pod \"payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.716889 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.716855    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:01.733213 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.733168    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:08:01.733954 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:08:01.733926    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8b7146_727c_4bb7_a9ae_e707cb3856af.slice/crio-e9bf585efc69772f76c05a21702214dc1804c18852b1ff74464a111e8ab005e8 WatchSource:0}: Error finding container e9bf585efc69772f76c05a21702214dc1804c18852b1ff74464a111e8ab005e8: Status 404 returned error can't find the container with id e9bf585efc69772f76c05a21702214dc1804c18852b1ff74464a111e8ab005e8
Sep 17 22:08:01.735475 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.735457    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 22:08:01.996432 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:01.996404    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:08:01.998393 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:08:01.998367    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222094e1_68e1_4a59_b867_0feb7c16f1d5.slice/crio-a00e9f4dd8c61010e1155739621640074989e17587e87571b5eb9c0b8dc8b8af WatchSource:0}: Error finding container a00e9f4dd8c61010e1155739621640074989e17587e87571b5eb9c0b8dc8b8af: Status 404 returned error can't find the container with id a00e9f4dd8c61010e1155739621640074989e17587e87571b5eb9c0b8dc8b8af
Sep 17 22:08:02.089937 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:02.089898    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" event={"ID":"cf8b7146-727c-4bb7-a9ae-e707cb3856af","Type":"ContainerStarted","Data":"e9bf585efc69772f76c05a21702214dc1804c18852b1ff74464a111e8ab005e8"}
Sep 17 22:08:02.091827 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:02.091789    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" event={"ID":"222094e1-68e1-4a59-b867-0feb7c16f1d5","Type":"ContainerStarted","Data":"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"}
Sep 17 22:08:02.091968 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:02.091843    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" event={"ID":"222094e1-68e1-4a59-b867-0feb7c16f1d5","Type":"ContainerStarted","Data":"a00e9f4dd8c61010e1155739621640074989e17587e87571b5eb9c0b8dc8b8af"}
Sep 17 22:08:02.092020 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:02.091997    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:02.114845 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:02.114793    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" podStartSLOduration=1.114773897 podStartE2EDuration="1.114773897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:02.111941815 +0000 UTC m=+1585.460675698" watchObservedRunningTime="2026-09-17 22:08:02.114773897 +0000 UTC m=+1585.463507816"
Sep 17 22:08:03.099169 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:03.099131    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" event={"ID":"cf8b7146-727c-4bb7-a9ae-e707cb3856af","Type":"ContainerStarted","Data":"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"}
Sep 17 22:08:03.099568 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:03.099481    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:03.120346 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:03.120275    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" podStartSLOduration=2.664813017 podStartE2EDuration="3.120260306s" totalImagesPullingTime="455.447289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:03.118803423 +0000 UTC m=+1586.467537305" watchObservedRunningTime="2026-09-17 22:08:03.120260306 +0000 UTC m=+1586.468994185"
Sep 17 22:08:09.110350 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:09.110317    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:08:13.101492 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:13.101453    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:08:32.151771 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.151737    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:08:32.155602 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.155584    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.158987 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.158963    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7dd893-serving-cert\""
Sep 17 22:08:32.172509 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.172480    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:08:32.227788 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.227756    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.227964 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.227867    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98bkz\" (UniqueName: \"kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.227964 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.227925    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.228044 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.227972    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.329487 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.329450    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98bkz\" (UniqueName: \"kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.329671 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.329498    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.329671 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.329536    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.329671 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.329577    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.329817 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:08:32.329672    2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-serving-cert: secret "maas-api-e2e-api-b-7dd893-serving-cert" not found
Sep 17 22:08:32.329817 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:08:32.329734    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls podName:aaa53b6b-d65c-481b-8b93-a284c06ff27c nodeName:}" failed. No retries permitted until 2026-09-17 22:08:32.829718019 +0000 UTC m=+1616.178451877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls") pod "maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" (UID: "aaa53b6b-d65c-481b-8b93-a284c06ff27c") : secret "maas-api-e2e-api-b-7dd893-serving-cert" not found
Sep 17 22:08:32.330220 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.330191    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.332179 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.332156    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.344352 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.344334    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98bkz\" (UniqueName: \"kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.834802 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.834775    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.837374 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.837356    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") pod \"maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:32.973992 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.973955    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:08:32.977905 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.977890    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:32.982031 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.982009    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-7dd893-dockercfg-5ld6l\""
Sep 17 22:08:32.982770 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.982748    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-7dd893\""
Sep 17 22:08:32.998794 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:32.998768    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:08:33.065750 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.065714    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:33.139154 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.139109    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwpwk\" (UniqueName: \"kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.139343 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.139203    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.240399 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.240357    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwpwk\" (UniqueName: \"kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.240826 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.240426    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.241237 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.241216    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.251628 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.251601    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwpwk\" (UniqueName: \"kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk\") pod \"payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.289086 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.289051    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:33.348711 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.348627    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:08:33.571092 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:33.571068    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:08:33.573100 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:08:33.573071    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79026f29_cdf2_49b4_bccc_d486e1d09a42.slice/crio-ebf01e6960d1afc82e9d1428cb795325cd73fc9d9be1ae43b8dd0468fc9a76a0 WatchSource:0}: Error finding container ebf01e6960d1afc82e9d1428cb795325cd73fc9d9be1ae43b8dd0468fc9a76a0: Status 404 returned error can't find the container with id ebf01e6960d1afc82e9d1428cb795325cd73fc9d9be1ae43b8dd0468fc9a76a0
Sep 17 22:08:34.274941 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.274904    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" event={"ID":"79026f29-cdf2-49b4-bccc-d486e1d09a42","Type":"ContainerStarted","Data":"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"}
Sep 17 22:08:34.275419 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.274967    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" event={"ID":"79026f29-cdf2-49b4-bccc-d486e1d09a42","Type":"ContainerStarted","Data":"ebf01e6960d1afc82e9d1428cb795325cd73fc9d9be1ae43b8dd0468fc9a76a0"}
Sep 17 22:08:34.275419 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.274995    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:34.276663 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.276638    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" event={"ID":"aaa53b6b-d65c-481b-8b93-a284c06ff27c","Type":"ContainerStarted","Data":"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"}
Sep 17 22:08:34.276770 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.276684    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" event={"ID":"aaa53b6b-d65c-481b-8b93-a284c06ff27c","Type":"ContainerStarted","Data":"e6347f2e687410d1bce22b886fe651310f74c519b5324ca35451d5a6ee2e9dd1"}
Sep 17 22:08:34.276770 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.276706    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:34.300305 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.300223    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" podStartSLOduration=2.300202598 podStartE2EDuration="2.300202598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:34.29659535 +0000 UTC m=+1617.645329230" watchObservedRunningTime="2026-09-17 22:08:34.300202598 +0000 UTC m=+1617.648936479"
Sep 17 22:08:34.322168 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:34.322111    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" podStartSLOduration=1.9885184470000001 podStartE2EDuration="2.322095372s" totalImagesPullingTime="333.576925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:34.319454255 +0000 UTC m=+1617.668188148" watchObservedRunningTime="2026-09-17 22:08:34.322095372 +0000 UTC m=+1617.670829289"
Sep 17 22:08:40.289718 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:40.289681    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:08:45.286754 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:45.286722    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:08:50.382857 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:08:50.382818    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 22:09:05.404638 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.404587    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" podUID="ce6cc4dc-3600-4717-9795-af4c6dddf674" containerName="main" containerID="cri-o://26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484" gracePeriod=45
Sep 17 22:09:05.424373 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.424344    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 22:09:05.424581 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.424561    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" podUID="583517c8-0e05-4e10-bb7f-886806dfaab2" containerName="maas-api" containerID="cri-o://e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b" gracePeriod=30
Sep 17 22:09:05.502343 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.502302    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 22:09:05.502633 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.502575    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" podUID="443259d0-308d-4f82-9d61-6ae6db6bb569" containerName="payload-processing" containerID="cri-o://7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f" gracePeriod=30
Sep 17 22:09:05.690336 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.690310    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 22:09:05.782788 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.782762    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 22:09:05.786331 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786263    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786465 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786365    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786519 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786503    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4nmk\" (UniqueName: \"kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786519 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786500    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:09:05.786657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786542    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786570    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786590    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:09:05.786657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786642    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs\") pod \"ce6cc4dc-3600-4717-9795-af4c6dddf674\" (UID: \"ce6cc4dc-3600-4717-9795-af4c6dddf674\") "
Sep 17 22:09:05.786895 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786811    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home" (OuterVolumeSpecName: "home") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:09:05.786999 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.786981    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 22:09:05.787149 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.787126    2587 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-home\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.787269 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.787153    2587 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-tmp-dir\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.787269 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.787167    2587 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-model-cache\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.789551 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.789506    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk" (OuterVolumeSpecName: "kube-api-access-g4nmk") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "kube-api-access-g4nmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:05.789669 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.789619    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm" (OuterVolumeSpecName: "dshm") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:09:05.789669 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.789620    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce6cc4dc-3600-4717-9795-af4c6dddf674" (UID: "ce6cc4dc-3600-4717-9795-af4c6dddf674"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:05.888256 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888226    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") pod \"583517c8-0e05-4e10-bb7f-886806dfaab2\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") "
Sep 17 22:09:05.888433 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888300    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz9lg\" (UniqueName: \"kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg\") pod \"583517c8-0e05-4e10-bb7f-886806dfaab2\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") "
Sep 17 22:09:05.888433 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888334    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf7cn\" (UniqueName: \"kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn\") pod \"443259d0-308d-4f82-9d61-6ae6db6bb569\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") "
Sep 17 22:09:05.888433 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888364    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume\") pod \"443259d0-308d-4f82-9d61-6ae6db6bb569\" (UID: \"443259d0-308d-4f82-9d61-6ae6db6bb569\") "
Sep 17 22:09:05.888433 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888411    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca\") pod \"583517c8-0e05-4e10-bb7f-886806dfaab2\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") "
Sep 17 22:09:05.888633 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888478    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs\") pod \"583517c8-0e05-4e10-bb7f-886806dfaab2\" (UID: \"583517c8-0e05-4e10-bb7f-886806dfaab2\") "
Sep 17 22:09:05.888798 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888774    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "583517c8-0e05-4e10-bb7f-886806dfaab2" (UID: "583517c8-0e05-4e10-bb7f-886806dfaab2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:05.888798 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888798    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "443259d0-308d-4f82-9d61-6ae6db6bb569" (UID: "443259d0-308d-4f82-9d61-6ae6db6bb569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:05.888935 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888858    2587 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce6cc4dc-3600-4717-9795-af4c6dddf674-dshm\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.888935 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888895    2587 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce6cc4dc-3600-4717-9795-af4c6dddf674-tls-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.888935 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.888910    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4nmk\" (UniqueName: \"kubernetes.io/projected/ce6cc4dc-3600-4717-9795-af4c6dddf674-kube-api-access-g4nmk\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.890847 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.890820    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "583517c8-0e05-4e10-bb7f-886806dfaab2" (UID: "583517c8-0e05-4e10-bb7f-886806dfaab2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:05.890933 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.890825    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn" (OuterVolumeSpecName: "kube-api-access-nf7cn") pod "443259d0-308d-4f82-9d61-6ae6db6bb569" (UID: "443259d0-308d-4f82-9d61-6ae6db6bb569"). InnerVolumeSpecName "kube-api-access-nf7cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:05.891051 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.891034    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "583517c8-0e05-4e10-bb7f-886806dfaab2" (UID: "583517c8-0e05-4e10-bb7f-886806dfaab2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:05.891097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.891054    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg" (OuterVolumeSpecName: "kube-api-access-bz9lg") pod "583517c8-0e05-4e10-bb7f-886806dfaab2" (UID: "583517c8-0e05-4e10-bb7f-886806dfaab2"). InnerVolumeSpecName "kube-api-access-bz9lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:05.990501 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990465    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443259d0-308d-4f82-9d61-6ae6db6bb569-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.990501 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990494    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583517c8-0e05-4e10-bb7f-886806dfaab2-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.990501 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990504    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.990738 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990512    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583517c8-0e05-4e10-bb7f-886806dfaab2-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.990738 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990521    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz9lg\" (UniqueName: \"kubernetes.io/projected/583517c8-0e05-4e10-bb7f-886806dfaab2-kube-api-access-bz9lg\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.990738 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:05.990531    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf7cn\" (UniqueName: \"kubernetes.io/projected/443259d0-308d-4f82-9d61-6ae6db6bb569-kube-api-access-nf7cn\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:06.469621 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.469585    2587 generic.go:356] "Generic (PLEG): container finished" podID="583517c8-0e05-4e10-bb7f-886806dfaab2" containerID="e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b" exitCode=0
Sep 17 22:09:06.470050 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.469629    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"
Sep 17 22:09:06.470050 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.469671    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" event={"ID":"583517c8-0e05-4e10-bb7f-886806dfaab2","Type":"ContainerDied","Data":"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"}
Sep 17 22:09:06.470050 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.469727    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf" event={"ID":"583517c8-0e05-4e10-bb7f-886806dfaab2","Type":"ContainerDied","Data":"34ef0e925fe274014b5c6947fd298c2a59e2052ed4b323f19b78325babba0223"}
Sep 17 22:09:06.470050 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.469764    2587 scope.go:117] "RemoveContainer" containerID="e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"
Sep 17 22:09:06.471217 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.471155    2587 generic.go:356] "Generic (PLEG): container finished" podID="443259d0-308d-4f82-9d61-6ae6db6bb569" containerID="7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f" exitCode=0
Sep 17 22:09:06.471217 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.471202    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"
Sep 17 22:09:06.471357 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.471267    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" event={"ID":"443259d0-308d-4f82-9d61-6ae6db6bb569","Type":"ContainerDied","Data":"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"}
Sep 17 22:09:06.471357 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.471337    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt" event={"ID":"443259d0-308d-4f82-9d61-6ae6db6bb569","Type":"ContainerDied","Data":"1863202c9c7e3de16548648c5577f644af2380a0bdf52839197e2d6de4a37f0d"}
Sep 17 22:09:06.472807 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.472776    2587 generic.go:356] "Generic (PLEG): container finished" podID="ce6cc4dc-3600-4717-9795-af4c6dddf674" containerID="26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484" exitCode=0
Sep 17 22:09:06.472929 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.472814    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"
Sep 17 22:09:06.472929 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.472826    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" event={"ID":"ce6cc4dc-3600-4717-9795-af4c6dddf674","Type":"ContainerDied","Data":"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"}
Sep 17 22:09:06.472929 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.472850    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj" event={"ID":"ce6cc4dc-3600-4717-9795-af4c6dddf674","Type":"ContainerDied","Data":"355b78beedff959a0200665ed73fd287c86158dd86220b9a2945c034a58f7015"}
Sep 17 22:09:06.481698 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.481682    2587 scope.go:117] "RemoveContainer" containerID="e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"
Sep 17 22:09:06.481985 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:06.481961    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b\": container with ID starting with e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b not found: ID does not exist" containerID="e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"
Sep 17 22:09:06.482077 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.481995    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b"} err="failed to get container status \"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b\": rpc error: code = NotFound desc = could not find container \"e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b\": container with ID starting with e39d43d26ea075502f0fa77fb9f97aa83ac6eabef96a44828915915308b7568b not found: ID does not exist"
Sep 17 22:09:06.482077 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.482019    2587 scope.go:117] "RemoveContainer" containerID="7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"
Sep 17 22:09:06.491149 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.491058    2587 scope.go:117] "RemoveContainer" containerID="7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"
Sep 17 22:09:06.494507 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:06.494478    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f\": container with ID starting with 7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f not found: ID does not exist" containerID="7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"
Sep 17 22:09:06.494641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.494515    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f"} err="failed to get container status \"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f\": rpc error: code = NotFound desc = could not find container \"7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f\": container with ID starting with 7ec71adc087950c7ad964a27bae71c42413561ce02a8c0bbc610b4a831b0f78f not found: ID does not exist"
Sep 17 22:09:06.494641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.494533    2587 scope.go:117] "RemoveContainer" containerID="26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"
Sep 17 22:09:06.503084 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.503068    2587 scope.go:117] "RemoveContainer" containerID="26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"
Sep 17 22:09:06.503386 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:06.503360    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484\": container with ID starting with 26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484 not found: ID does not exist" containerID="26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"
Sep 17 22:09:06.503460 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.503393    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484"} err="failed to get container status \"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484\": rpc error: code = NotFound desc = could not find container \"26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484\": container with ID starting with 26d07ce950232ae61ef5cca2237c7c01559c0a89d8271274d4af6a14f50f4484 not found: ID does not exist"
Sep 17 22:09:06.529015 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.528986    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 22:09:06.550262 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.550233    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f996ae-6778c9b7fb-hsktf"]
Sep 17 22:09:06.578725 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.578693    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 22:09:06.589688 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.589657    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f996ae-6ff6f44479-wh7kt"]
Sep 17 22:09:06.612872 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.612841    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 22:09:06.618663 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.618639    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f996ae/premium-w0-f996ae-kserve-7b6b5cd74d-kzlhj"]
Sep 17 22:09:06.858336 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.858305    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="443259d0-308d-4f82-9d61-6ae6db6bb569" path="/var/lib/kubelet/pods/443259d0-308d-4f82-9d61-6ae6db6bb569/volumes"
Sep 17 22:09:06.858744 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.858729    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="583517c8-0e05-4e10-bb7f-886806dfaab2" path="/var/lib/kubelet/pods/583517c8-0e05-4e10-bb7f-886806dfaab2/volumes"
Sep 17 22:09:06.859089 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:06.859078    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce6cc4dc-3600-4717-9795-af4c6dddf674" path="/var/lib/kubelet/pods/ce6cc4dc-3600-4717-9795-af4c6dddf674/volumes"
Sep 17 22:09:13.904998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:13.904965    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:09:13.905536 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:13.905159    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerName="maas-api" containerID="cri-o://11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee" gracePeriod=30
Sep 17 22:09:13.991615 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:13.991583    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:09:13.991799 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:13.991777    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" podUID="222094e1-68e1-4a59-b867-0feb7c16f1d5" containerName="payload-pre-processing" containerID="cri-o://123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7" gracePeriod=30
Sep 17 22:09:14.105441 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.105326    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.63:8443/health\": dial tcp 10.134.0.63:8443: connect: connection refused"
Sep 17 22:09:14.281456 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.281416    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:09:14.285467 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.285446    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:09:14.380851 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.380816    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxftj\" (UniqueName: \"kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj\") pod \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") "
Sep 17 22:09:14.381007 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.380921    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs\") pod \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") "
Sep 17 22:09:14.381007 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.380945    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca\") pod \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") "
Sep 17 22:09:14.381007 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.380983    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftrs8\" (UniqueName: \"kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8\") pod \"222094e1-68e1-4a59-b867-0feb7c16f1d5\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") "
Sep 17 22:09:14.381133 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.381032    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") pod \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\" (UID: \"cf8b7146-727c-4bb7-a9ae-e707cb3856af\") "
Sep 17 22:09:14.381133 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.381051    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume\") pod \"222094e1-68e1-4a59-b867-0feb7c16f1d5\" (UID: \"222094e1-68e1-4a59-b867-0feb7c16f1d5\") "
Sep 17 22:09:14.381370 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.381344    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf8b7146-727c-4bb7-a9ae-e707cb3856af" (UID: "cf8b7146-727c-4bb7-a9ae-e707cb3856af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:14.381575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.381549    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "222094e1-68e1-4a59-b867-0feb7c16f1d5" (UID: "222094e1-68e1-4a59-b867-0feb7c16f1d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:14.383263 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.383240    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj" (OuterVolumeSpecName: "kube-api-access-cxftj") pod "cf8b7146-727c-4bb7-a9ae-e707cb3856af" (UID: "cf8b7146-727c-4bb7-a9ae-e707cb3856af"). InnerVolumeSpecName "kube-api-access-cxftj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:14.383381 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.383262    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8" (OuterVolumeSpecName: "kube-api-access-ftrs8") pod "222094e1-68e1-4a59-b867-0feb7c16f1d5" (UID: "222094e1-68e1-4a59-b867-0feb7c16f1d5"). InnerVolumeSpecName "kube-api-access-ftrs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:14.383735 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.383711    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf8b7146-727c-4bb7-a9ae-e707cb3856af" (UID: "cf8b7146-727c-4bb7-a9ae-e707cb3856af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:14.383735 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.383724    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf8b7146-727c-4bb7-a9ae-e707cb3856af" (UID: "cf8b7146-727c-4bb7-a9ae-e707cb3856af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:14.482087 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482054    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.482087 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482084    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf8b7146-727c-4bb7-a9ae-e707cb3856af-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.482319 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482094    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftrs8\" (UniqueName: \"kubernetes.io/projected/222094e1-68e1-4a59-b867-0feb7c16f1d5-kube-api-access-ftrs8\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.482319 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482105    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf8b7146-727c-4bb7-a9ae-e707cb3856af-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.482319 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482114    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/222094e1-68e1-4a59-b867-0feb7c16f1d5-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.482319 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.482123    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxftj\" (UniqueName: \"kubernetes.io/projected/cf8b7146-727c-4bb7-a9ae-e707cb3856af-kube-api-access-cxftj\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:14.523654 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.523623    2587 generic.go:356] "Generic (PLEG): container finished" podID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerID="11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee" exitCode=0
Sep 17 22:09:14.523825 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.523666    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"
Sep 17 22:09:14.523825 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.523718    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" event={"ID":"cf8b7146-727c-4bb7-a9ae-e707cb3856af","Type":"ContainerDied","Data":"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"}
Sep 17 22:09:14.523825 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.523773    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k" event={"ID":"cf8b7146-727c-4bb7-a9ae-e707cb3856af","Type":"ContainerDied","Data":"e9bf585efc69772f76c05a21702214dc1804c18852b1ff74464a111e8ab005e8"}
Sep 17 22:09:14.523825 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.523802    2587 scope.go:117] "RemoveContainer" containerID="11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"
Sep 17 22:09:14.525225 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.525199    2587 generic.go:356] "Generic (PLEG): container finished" podID="222094e1-68e1-4a59-b867-0feb7c16f1d5" containerID="123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7" exitCode=0
Sep 17 22:09:14.525360 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.525252    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" event={"ID":"222094e1-68e1-4a59-b867-0feb7c16f1d5","Type":"ContainerDied","Data":"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"}
Sep 17 22:09:14.525360 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.525257    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"
Sep 17 22:09:14.525360 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.525307    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s" event={"ID":"222094e1-68e1-4a59-b867-0feb7c16f1d5","Type":"ContainerDied","Data":"a00e9f4dd8c61010e1155739621640074989e17587e87571b5eb9c0b8dc8b8af"}
Sep 17 22:09:14.534896 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.534801    2587 scope.go:117] "RemoveContainer" containerID="11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"
Sep 17 22:09:14.535179 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:14.535150    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee\": container with ID starting with 11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee not found: ID does not exist" containerID="11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"
Sep 17 22:09:14.535306 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.535181    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee"} err="failed to get container status \"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee\": rpc error: code = NotFound desc = could not find container \"11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee\": container with ID starting with 11fa5a426f4889ce045bf793c134ab71dcd278160c3f18c2119cd6eb154293ee not found: ID does not exist"
Sep 17 22:09:14.535306 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.535199    2587 scope.go:117] "RemoveContainer" containerID="123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"
Sep 17 22:09:14.545109 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.545092    2587 scope.go:117] "RemoveContainer" containerID="123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"
Sep 17 22:09:14.545384 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:14.545363    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7\": container with ID starting with 123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7 not found: ID does not exist" containerID="123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"
Sep 17 22:09:14.545448 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.545394    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7"} err="failed to get container status \"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7\": rpc error: code = NotFound desc = could not find container \"123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7\": container with ID starting with 123d2742d75c46a3b4e311ac944bb342d0eb78668720e5531d3ebab1ac9debf7 not found: ID does not exist"
Sep 17 22:09:14.560718 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.560686    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:09:14.567300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.567260    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-194000-6b89896cd6-nbg9s"]
Sep 17 22:09:14.583716 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.583681    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:09:14.587864 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.587839    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-194000-86f845648b-9w46k"]
Sep 17 22:09:14.858748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.858662    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222094e1-68e1-4a59-b867-0feb7c16f1d5" path="/var/lib/kubelet/pods/222094e1-68e1-4a59-b867-0feb7c16f1d5/volumes"
Sep 17 22:09:14.859086 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:14.859072    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" path="/var/lib/kubelet/pods/cf8b7146-727c-4bb7-a9ae-e707cb3856af/volumes"
Sep 17 22:09:40.265019 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.264977    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:09:40.265553 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.265426    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerName="maas-api" containerID="cri-o://f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60" gracePeriod=30
Sep 17 22:09:40.284085 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.284043    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.65:8443/health\": dial tcp 10.134.0.65:8443: connect: connection refused"
Sep 17 22:09:40.329413 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.329375    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:09:40.329699 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.329643    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" podUID="79026f29-cdf2-49b4-bccc-d486e1d09a42" containerName="payload-pre-processing" containerID="cri-o://465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874" gracePeriod=30
Sep 17 22:09:40.596080 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.596051    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:09:40.625640 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.625613    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:09:40.674365 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.674323    2587 generic.go:356] "Generic (PLEG): container finished" podID="79026f29-cdf2-49b4-bccc-d486e1d09a42" containerID="465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874" exitCode=0
Sep 17 22:09:40.674575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.674415    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"
Sep 17 22:09:40.674575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.674434    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" event={"ID":"79026f29-cdf2-49b4-bccc-d486e1d09a42","Type":"ContainerDied","Data":"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"}
Sep 17 22:09:40.674575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.674493    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw" event={"ID":"79026f29-cdf2-49b4-bccc-d486e1d09a42","Type":"ContainerDied","Data":"ebf01e6960d1afc82e9d1428cb795325cd73fc9d9be1ae43b8dd0468fc9a76a0"}
Sep 17 22:09:40.674575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.674522    2587 scope.go:117] "RemoveContainer" containerID="465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"
Sep 17 22:09:40.676128 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.676090    2587 generic.go:356] "Generic (PLEG): container finished" podID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerID="f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60" exitCode=0
Sep 17 22:09:40.676428 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.676154    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"
Sep 17 22:09:40.676428 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.676164    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" event={"ID":"aaa53b6b-d65c-481b-8b93-a284c06ff27c","Type":"ContainerDied","Data":"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"}
Sep 17 22:09:40.676428 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.676210    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr" event={"ID":"aaa53b6b-d65c-481b-8b93-a284c06ff27c","Type":"ContainerDied","Data":"e6347f2e687410d1bce22b886fe651310f74c519b5324ca35451d5a6ee2e9dd1"}
Sep 17 22:09:40.684878 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.684853    2587 scope.go:117] "RemoveContainer" containerID="465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"
Sep 17 22:09:40.685189 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:40.685167    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874\": container with ID starting with 465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874 not found: ID does not exist" containerID="465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"
Sep 17 22:09:40.685244 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.685199    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874"} err="failed to get container status \"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874\": rpc error: code = NotFound desc = could not find container \"465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874\": container with ID starting with 465670d33f5e0594c87cf05f1a5aaf74ca6d0a09316049e0dbcf34f03f987874 not found: ID does not exist"
Sep 17 22:09:40.685244 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.685219    2587 scope.go:117] "RemoveContainer" containerID="f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"
Sep 17 22:09:40.693973 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.693954    2587 scope.go:117] "RemoveContainer" containerID="f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"
Sep 17 22:09:40.694297 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:09:40.694266    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60\": container with ID starting with f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60 not found: ID does not exist" containerID="f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"
Sep 17 22:09:40.694359 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.694312    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60"} err="failed to get container status \"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60\": rpc error: code = NotFound desc = could not find container \"f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60\": container with ID starting with f9e91e1b0ebc0cac63e1039e4f3d2b7bd6e4efa90e8365b60718828dd9603c60 not found: ID does not exist"
Sep 17 22:09:40.745520 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745484    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") pod \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") "
Sep 17 22:09:40.745709 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745529    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwpwk\" (UniqueName: \"kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk\") pod \"79026f29-cdf2-49b4-bccc-d486e1d09a42\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") "
Sep 17 22:09:40.745709 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745591    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca\") pod \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") "
Sep 17 22:09:40.745709 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745627    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98bkz\" (UniqueName: \"kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz\") pod \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") "
Sep 17 22:09:40.745709 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745687    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs\") pod \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\" (UID: \"aaa53b6b-d65c-481b-8b93-a284c06ff27c\") "
Sep 17 22:09:40.745918 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745764    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume\") pod \"79026f29-cdf2-49b4-bccc-d486e1d09a42\" (UID: \"79026f29-cdf2-49b4-bccc-d486e1d09a42\") "
Sep 17 22:09:40.745984 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.745940    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aaa53b6b-d65c-481b-8b93-a284c06ff27c" (UID: "aaa53b6b-d65c-481b-8b93-a284c06ff27c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:40.746435 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.746362    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79026f29-cdf2-49b4-bccc-d486e1d09a42" (UID: "79026f29-cdf2-49b4-bccc-d486e1d09a42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:40.746635 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.746588    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa53b6b-d65c-481b-8b93-a284c06ff27c-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:40.746635 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.746615    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79026f29-cdf2-49b4-bccc-d486e1d09a42-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:40.747955 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.747932    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aaa53b6b-d65c-481b-8b93-a284c06ff27c" (UID: "aaa53b6b-d65c-481b-8b93-a284c06ff27c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:40.748309 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.748274    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz" (OuterVolumeSpecName: "kube-api-access-98bkz") pod "aaa53b6b-d65c-481b-8b93-a284c06ff27c" (UID: "aaa53b6b-d65c-481b-8b93-a284c06ff27c"). InnerVolumeSpecName "kube-api-access-98bkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:40.748398 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.748327    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aaa53b6b-d65c-481b-8b93-a284c06ff27c" (UID: "aaa53b6b-d65c-481b-8b93-a284c06ff27c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:40.748605 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.748580    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk" (OuterVolumeSpecName: "kube-api-access-qwpwk") pod "79026f29-cdf2-49b4-bccc-d486e1d09a42" (UID: "79026f29-cdf2-49b4-bccc-d486e1d09a42"). InnerVolumeSpecName "kube-api-access-qwpwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:09:40.847464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.847366    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:40.847464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.847408    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwpwk\" (UniqueName: \"kubernetes.io/projected/79026f29-cdf2-49b4-bccc-d486e1d09a42-kube-api-access-qwpwk\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:40.847464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.847423    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98bkz\" (UniqueName: \"kubernetes.io/projected/aaa53b6b-d65c-481b-8b93-a284c06ff27c-kube-api-access-98bkz\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:40.847464 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:40.847435    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa53b6b-d65c-481b-8b93-a284c06ff27c-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:09:41.002630 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:41.002587    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:09:41.011259 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:41.011212    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7dd893-6ffc85b4c8-pxsgw"]
Sep 17 22:09:41.026819 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:41.026764    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:09:41.029754 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:41.029721    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7dd893-759ffbcdf6-j4sqr"]
Sep 17 22:09:42.857888 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:42.857854    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79026f29-cdf2-49b4-bccc-d486e1d09a42" path="/var/lib/kubelet/pods/79026f29-cdf2-49b4-bccc-d486e1d09a42/volumes"
Sep 17 22:09:42.858278 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:42.858265    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" path="/var/lib/kubelet/pods/aaa53b6b-d65c-481b-8b93-a284c06ff27c/volumes"
Sep 17 22:09:59.904824 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:59.904741    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:09:59.905356 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:09:59.904962    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" podUID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" containerName="payload-processing" containerID="cri-o://5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b" gracePeriod=30
Sep 17 22:10:00.175503 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.175472    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:10:00.240121 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.240086    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume\") pod \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") "
Sep 17 22:10:00.240354 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.240144    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf4t7\" (UniqueName: \"kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7\") pod \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\" (UID: \"39ee5a3c-c72a-4231-8802-4cd66f8804b4\") "
Sep 17 22:10:00.240622 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.240592    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39ee5a3c-c72a-4231-8802-4cd66f8804b4" (UID: "39ee5a3c-c72a-4231-8802-4cd66f8804b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:10:00.242493 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.242465    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7" (OuterVolumeSpecName: "kube-api-access-gf4t7") pod "39ee5a3c-c72a-4231-8802-4cd66f8804b4" (UID: "39ee5a3c-c72a-4231-8802-4cd66f8804b4"). InnerVolumeSpecName "kube-api-access-gf4t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:10:00.341412 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.341368    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ee5a3c-c72a-4231-8802-4cd66f8804b4-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:00.341412 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.341404    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4t7\" (UniqueName: \"kubernetes.io/projected/39ee5a3c-c72a-4231-8802-4cd66f8804b4-kube-api-access-gf4t7\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:00.788504 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.788469    2587 generic.go:356] "Generic (PLEG): container finished" podID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" containerID="5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b" exitCode=0
Sep 17 22:10:00.788710 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.788513    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"
Sep 17 22:10:00.788710 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.788558    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" event={"ID":"39ee5a3c-c72a-4231-8802-4cd66f8804b4","Type":"ContainerDied","Data":"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"}
Sep 17 22:10:00.788710 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.788618    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6" event={"ID":"39ee5a3c-c72a-4231-8802-4cd66f8804b4","Type":"ContainerDied","Data":"c7ad2e58618d1f6c59c3ecad93d144945a3108f2476130a537bb2d4bab1e818c"}
Sep 17 22:10:00.788710 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.788647    2587 scope.go:117] "RemoveContainer" containerID="5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"
Sep 17 22:10:00.798339 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.798315    2587 scope.go:117] "RemoveContainer" containerID="5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"
Sep 17 22:10:00.798684 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:10:00.798651    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b\": container with ID starting with 5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b not found: ID does not exist" containerID="5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"
Sep 17 22:10:00.798755 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.798688    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b"} err="failed to get container status \"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b\": rpc error: code = NotFound desc = could not find container \"5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b\": container with ID starting with 5e8fef28e6eb57c45562019982e322f84b0c3b0b98ee96ee3d9f663978cb161b not found: ID does not exist"
Sep 17 22:10:00.832504 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.832470    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:10:00.842334 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.842302    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a23afc-b6b947f69-l2mr6"]
Sep 17 22:10:00.858494 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:00.858454    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" path="/var/lib/kubelet/pods/39ee5a3c-c72a-4231-8802-4cd66f8804b4/volumes"
Sep 17 22:10:42.505117 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:42.505079    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 22:10:53.555248 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.554803    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 22:10:53.555797 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.555605    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" podUID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" containerName="maas-api" containerID="cri-o://bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f" gracePeriod=30
Sep 17 22:10:53.644386 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.644349    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 22:10:53.644640 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.644613    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" podUID="81361305-1d90-4ab7-a7a4-3f1f138436c8" containerName="payload-processing" containerID="cri-o://1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef" gracePeriod=30
Sep 17 22:10:53.907257 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.907226    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 22:10:53.983115 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.983083    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume\") pod \"81361305-1d90-4ab7-a7a4-3f1f138436c8\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") "
Sep 17 22:10:53.983273 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.983168    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwt58\" (UniqueName: \"kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58\") pod \"81361305-1d90-4ab7-a7a4-3f1f138436c8\" (UID: \"81361305-1d90-4ab7-a7a4-3f1f138436c8\") "
Sep 17 22:10:53.983505 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.983478    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81361305-1d90-4ab7-a7a4-3f1f138436c8" (UID: "81361305-1d90-4ab7-a7a4-3f1f138436c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:10:53.983609 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.983598    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81361305-1d90-4ab7-a7a4-3f1f138436c8-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:53.985554 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:53.985527    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58" (OuterVolumeSpecName: "kube-api-access-zwt58") pod "81361305-1d90-4ab7-a7a4-3f1f138436c8" (UID: "81361305-1d90-4ab7-a7a4-3f1f138436c8"). InnerVolumeSpecName "kube-api-access-zwt58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:10:54.016847 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.016819    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 22:10:54.078465 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.078360    2587 generic.go:356] "Generic (PLEG): container finished" podID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" containerID="bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f" exitCode=0
Sep 17 22:10:54.078465 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.078411    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"
Sep 17 22:10:54.078465 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.078452    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" event={"ID":"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8","Type":"ContainerDied","Data":"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"}
Sep 17 22:10:54.078755 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.078512    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk" event={"ID":"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8","Type":"ContainerDied","Data":"fa8245bf70cfe374c7101e1e4a34fb67a276b6d23f91f2195d757fea1c0a4f85"}
Sep 17 22:10:54.078755 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.078542    2587 scope.go:117] "RemoveContainer" containerID="bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"
Sep 17 22:10:54.080169 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.079840    2587 generic.go:356] "Generic (PLEG): container finished" podID="81361305-1d90-4ab7-a7a4-3f1f138436c8" containerID="1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef" exitCode=0
Sep 17 22:10:54.080169 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.079875    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" event={"ID":"81361305-1d90-4ab7-a7a4-3f1f138436c8","Type":"ContainerDied","Data":"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"}
Sep 17 22:10:54.080169 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.079899    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"
Sep 17 22:10:54.080169 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.079936    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p" event={"ID":"81361305-1d90-4ab7-a7a4-3f1f138436c8","Type":"ContainerDied","Data":"1fd21b4300d4855d12422071ea925ab3eba746427238228e2e5fc83bd8b27761"}
Sep 17 22:10:54.084239 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.084215    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwt58\" (UniqueName: \"kubernetes.io/projected/81361305-1d90-4ab7-a7a4-3f1f138436c8-kube-api-access-zwt58\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:54.088432 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.088410    2587 scope.go:117] "RemoveContainer" containerID="bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"
Sep 17 22:10:54.088739 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:10:54.088719    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f\": container with ID starting with bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f not found: ID does not exist" containerID="bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"
Sep 17 22:10:54.088800 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.088750    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f"} err="failed to get container status \"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f\": rpc error: code = NotFound desc = could not find container \"bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f\": container with ID starting with bedfd117698fe01847397fdf887c968f36cc25a73a675868b3344fade0eb352f not found: ID does not exist"
Sep 17 22:10:54.088800 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.088766    2587 scope.go:117] "RemoveContainer" containerID="1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"
Sep 17 22:10:54.096934 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.096905    2587 scope.go:117] "RemoveContainer" containerID="1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"
Sep 17 22:10:54.097197 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:10:54.097178    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef\": container with ID starting with 1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef not found: ID does not exist" containerID="1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"
Sep 17 22:10:54.097258 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.097207    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef"} err="failed to get container status \"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef\": rpc error: code = NotFound desc = could not find container \"1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef\": container with ID starting with 1288df7c3d9b12e9e1eddd364e3ac9f4dd2aa895803706354b0cf990d74a49ef not found: ID does not exist"
Sep 17 22:10:54.108391 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.108365    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 22:10:54.112639 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.112618    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dbe8a7-777b494f85-d5f9p"]
Sep 17 22:10:54.184793 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.184755    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") pod \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") "
Sep 17 22:10:54.184793 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.184802    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs\") pod \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") "
Sep 17 22:10:54.184999 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.184846    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6tqg\" (UniqueName: \"kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg\") pod \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") "
Sep 17 22:10:54.184999 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.184957    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca\") pod \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\" (UID: \"7c4b72a4-7945-4ce7-b62b-6f9fd89babe8\") "
Sep 17 22:10:54.185409 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.185379    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" (UID: "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:10:54.187121 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.187097    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg" (OuterVolumeSpecName: "kube-api-access-h6tqg") pod "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" (UID: "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8"). InnerVolumeSpecName "kube-api-access-h6tqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:10:54.187121 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.187102    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" (UID: "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:10:54.187249 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.187112    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" (UID: "7c4b72a4-7945-4ce7-b62b-6f9fd89babe8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:10:54.286416 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.286380    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:54.286416 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.286409    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:54.286416 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.286417    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:54.286416 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.286425    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6tqg\" (UniqueName: \"kubernetes.io/projected/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8-kube-api-access-h6tqg\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:54.416475 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.416437    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 22:10:54.420034 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.420005    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dbe8a7-59d4c89989-9xrgk"]
Sep 17 22:10:54.857900 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.857863    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" path="/var/lib/kubelet/pods/7c4b72a4-7945-4ce7-b62b-6f9fd89babe8/volumes"
Sep 17 22:10:54.858267 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:54.858238    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81361305-1d90-4ab7-a7a4-3f1f138436c8" path="/var/lib/kubelet/pods/81361305-1d90-4ab7-a7a4-3f1f138436c8/volumes"
Sep 17 22:10:57.525587 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.525541    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" podUID="e9be7797-ce44-4622-9013-03822f604568" containerName="main" containerID="cri-o://c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f" gracePeriod=45
Sep 17 22:10:57.779076 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.779008    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 22:10:57.926217 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926184    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926396 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926257    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926396 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926350    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926478 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926430    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np6jj\" (UniqueName: \"kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926478 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926457    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926555 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926489    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache\") pod \"e9be7797-ce44-4622-9013-03822f604568\" (UID: \"e9be7797-ce44-4622-9013-03822f604568\") "
Sep 17 22:10:57.926600 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926521    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home" (OuterVolumeSpecName: "home") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:10:57.926600 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926582    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:10:57.926806 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926790    2587 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-home\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:57.926875 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926810    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:10:57.926875 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.926829    2587 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-tmp-dir\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:57.928744 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.928720    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj" (OuterVolumeSpecName: "kube-api-access-np6jj") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "kube-api-access-np6jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:10:57.928839 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.928765    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:10:57.928839 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:57.928804    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm" (OuterVolumeSpecName: "dshm") pod "e9be7797-ce44-4622-9013-03822f604568" (UID: "e9be7797-ce44-4622-9013-03822f604568"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:10:58.028216 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.028172    2587 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-model-cache\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:58.028216 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.028207    2587 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9be7797-ce44-4622-9013-03822f604568-dshm\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:58.028216 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.028217    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np6jj\" (UniqueName: \"kubernetes.io/projected/e9be7797-ce44-4622-9013-03822f604568-kube-api-access-np6jj\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:58.028216 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.028228    2587 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9be7797-ce44-4622-9013-03822f604568-tls-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:10:58.106178 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.106087    2587 generic.go:356] "Generic (PLEG): container finished" podID="e9be7797-ce44-4622-9013-03822f604568" containerID="c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f" exitCode=0
Sep 17 22:10:58.106178 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.106133    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"
Sep 17 22:10:58.106460 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.106185    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" event={"ID":"e9be7797-ce44-4622-9013-03822f604568","Type":"ContainerDied","Data":"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"}
Sep 17 22:10:58.106460 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.106231    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps" event={"ID":"e9be7797-ce44-4622-9013-03822f604568","Type":"ContainerDied","Data":"60ec7ee9acdfa2ef8734ef53b53faf975ce3cde9d078ac011aa25ec8a69077e6"}
Sep 17 22:10:58.106460 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.106251    2587 scope.go:117] "RemoveContainer" containerID="c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"
Sep 17 22:10:58.116212 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.116196    2587 scope.go:117] "RemoveContainer" containerID="c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"
Sep 17 22:10:58.116572 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:10:58.116550    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f\": container with ID starting with c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f not found: ID does not exist" containerID="c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"
Sep 17 22:10:58.116660 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.116582    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f"} err="failed to get container status \"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f\": rpc error: code = NotFound desc = could not find container \"c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f\": container with ID starting with c85e63f61265437feb2f335b9031596ea7b58e68083fcd7abee12f3f928e8e1f not found: ID does not exist"
Sep 17 22:10:58.138489 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.138462    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 22:10:58.147130 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.147100    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-dbe8a7/premium-w4-dbe8a7-kserve-58ff6d5dfc-x92ps"]
Sep 17 22:10:58.856911 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:10:58.856876    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9be7797-ce44-4622-9013-03822f604568" path="/var/lib/kubelet/pods/e9be7797-ce44-4622-9013-03822f604568/volumes"
Sep 17 22:11:38.918986 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:11:38.918956    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:11:38.922796 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:11:38.922773    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:15:00.137949 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.137910    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"]
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138379    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerName="maas-api"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138390    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerName="maas-api"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138400    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81361305-1d90-4ab7-a7a4-3f1f138436c8" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138407    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="81361305-1d90-4ab7-a7a4-3f1f138436c8" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138416    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" containerName="maas-api"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138422    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" containerName="maas-api"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138429    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9be7797-ce44-4622-9013-03822f604568" containerName="main"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138434    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9be7797-ce44-4622-9013-03822f604568" containerName="main"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138445    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138450    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138460    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443259d0-308d-4f82-9d61-6ae6db6bb569" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138465    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="443259d0-308d-4f82-9d61-6ae6db6bb569" containerName="payload-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138473    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79026f29-cdf2-49b4-bccc-d486e1d09a42" containerName="payload-pre-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138478    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="79026f29-cdf2-49b4-bccc-d486e1d09a42" containerName="payload-pre-processing"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138490    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="583517c8-0e05-4e10-bb7f-886806dfaab2" containerName="maas-api"
Sep 17 22:15:00.138491 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138495    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="583517c8-0e05-4e10-bb7f-886806dfaab2" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138503    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222094e1-68e1-4a59-b867-0feb7c16f1d5" containerName="payload-pre-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138508    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="222094e1-68e1-4a59-b867-0feb7c16f1d5" containerName="payload-pre-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138514    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138521    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138532    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce6cc4dc-3600-4717-9795-af4c6dddf674" containerName="main"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138540    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce6cc4dc-3600-4717-9795-af4c6dddf674" containerName="main"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138630    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="443259d0-308d-4f82-9d61-6ae6db6bb569" containerName="payload-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138641    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="79026f29-cdf2-49b4-bccc-d486e1d09a42" containerName="payload-pre-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138650    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9be7797-ce44-4622-9013-03822f604568" containerName="main"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138659    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="583517c8-0e05-4e10-bb7f-886806dfaab2" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138666    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa53b6b-d65c-481b-8b93-a284c06ff27c" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138675    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce6cc4dc-3600-4717-9795-af4c6dddf674" containerName="main"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138680    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="222094e1-68e1-4a59-b867-0feb7c16f1d5" containerName="payload-pre-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138687    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="81361305-1d90-4ab7-a7a4-3f1f138436c8" containerName="payload-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138692    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf8b7146-727c-4bb7-a9ae-e707cb3856af" containerName="maas-api"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138699    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ee5a3c-c72a-4231-8802-4cd66f8804b4" containerName="payload-processing"
Sep 17 22:15:00.139300 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.138707    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4b72a4-7945-4ce7-b62b-6f9fd89babe8" containerName="maas-api"
Sep 17 22:15:00.142435 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.142415    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"
Sep 17 22:15:00.154771 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.154742    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmdbp\""
Sep 17 22:15:00.156454 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.156421    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"]
Sep 17 22:15:00.162700 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.162671    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"
Sep 17 22:15:00.444145 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:00.444113    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"]
Sep 17 22:15:00.446319 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:15:00.446263    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada5735b_e1f1_4702_bd01_b4e448aabd14.slice/crio-b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e WatchSource:0}: Error finding container b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e: Status 404 returned error can't find the container with id b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e
Sep 17 22:15:01.356943 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:01.356906    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k" event={"ID":"ada5735b-e1f1-4702-bd01-b4e448aabd14","Type":"ContainerStarted","Data":"bcca3a3061492bdce8b22cb576eb71d8d84ed8e7179cff4c5dcbcda6475e1804"}
Sep 17 22:15:01.357357 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:01.356969    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k" event={"ID":"ada5735b-e1f1-4702-bd01-b4e448aabd14","Type":"ContainerStarted","Data":"b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e"}
Sep 17 22:15:01.395414 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:01.395353    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k" podStartSLOduration=1.39533759 podStartE2EDuration="1.39533759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:15:01.393649637 +0000 UTC m=+2004.742383518" watchObservedRunningTime="2026-09-17 22:15:01.39533759 +0000 UTC m=+2004.744071470"
Sep 17 22:15:02.363734 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:02.363702    2587 generic.go:356] "Generic (PLEG): container finished" podID="ada5735b-e1f1-4702-bd01-b4e448aabd14" containerID="bcca3a3061492bdce8b22cb576eb71d8d84ed8e7179cff4c5dcbcda6475e1804" exitCode=0
Sep 17 22:15:02.364189 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:02.363759    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k" event={"ID":"ada5735b-e1f1-4702-bd01-b4e448aabd14","Type":"ContainerDied","Data":"bcca3a3061492bdce8b22cb576eb71d8d84ed8e7179cff4c5dcbcda6475e1804"}
Sep 17 22:15:03.515400 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:03.515376    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"
Sep 17 22:15:04.376597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:04.376566    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k"
Sep 17 22:15:04.376776 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:04.376609    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828055-jp48k" event={"ID":"ada5735b-e1f1-4702-bd01-b4e448aabd14","Type":"ContainerDied","Data":"b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e"}
Sep 17 22:15:04.376776 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:04.376656    2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5d3cb18b3181d19cc3d94fbdc0c017a7e296b69bc83dbb80677b852ee44d87e"
Sep 17 22:15:19.943665 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.943617    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:15:19.944102 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.944051    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada5735b-e1f1-4702-bd01-b4e448aabd14" containerName="cleanup"
Sep 17 22:15:19.944102 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.944064    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada5735b-e1f1-4702-bd01-b4e448aabd14" containerName="cleanup"
Sep 17 22:15:19.944175 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.944123    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada5735b-e1f1-4702-bd01-b4e448aabd14" containerName="cleanup"
Sep 17 22:15:19.947406 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.947387    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:19.951330 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.951309    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a71836-dockercfg-6xslr\""
Sep 17 22:15:19.953131 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.953112    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a71836\""
Sep 17 22:15:19.972253 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:19.972223    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:15:20.034353 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.034320    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:15:20.038228 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.038203    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.046922 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.046894    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ct7z\" (UniqueName: \"kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.047081 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.047030    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.057200 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.057165    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:15:20.148446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.148411    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58psh\" (UniqueName: \"kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.148446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.148460    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.148720 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.148485    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.148720 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.148586    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ct7z\" (UniqueName: \"kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.149130 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.149110    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.162123 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.162092    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ct7z\" (UniqueName: \"kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z\") pod \"payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.250024 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.249984    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58psh\" (UniqueName: \"kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.250240 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.250051    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.250830 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.250808    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.257348 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.257324    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:20.259936 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.259903    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58psh\" (UniqueName: \"kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh\") pod \"payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.349009 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.348801    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:20.536657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.536576    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:15:20.538973 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:15:20.538946    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefaaff4f_50a6_49a7_b146_17c6d8c9cf5c.slice/crio-e0136a9636ee8b66b4d79402181b1589e43e85831e9f519a6cb66237d2f0c5bc WatchSource:0}: Error finding container e0136a9636ee8b66b4d79402181b1589e43e85831e9f519a6cb66237d2f0c5bc: Status 404 returned error can't find the container with id e0136a9636ee8b66b4d79402181b1589e43e85831e9f519a6cb66237d2f0c5bc
Sep 17 22:15:20.627821 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:20.627612    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:15:20.631031 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:15:20.630998    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2d3ca5_f2de_4673_a02f_54931db1ced4.slice/crio-6acee398d5a7178c3a257cdf442ff0599f16d1a332d3513260c338e24680a364 WatchSource:0}: Error finding container 6acee398d5a7178c3a257cdf442ff0599f16d1a332d3513260c338e24680a364: Status 404 returned error can't find the container with id 6acee398d5a7178c3a257cdf442ff0599f16d1a332d3513260c338e24680a364
Sep 17 22:15:21.462457 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.462418    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" event={"ID":"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c","Type":"ContainerStarted","Data":"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"}
Sep 17 22:15:21.462901 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.462469    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" event={"ID":"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c","Type":"ContainerStarted","Data":"e0136a9636ee8b66b4d79402181b1589e43e85831e9f519a6cb66237d2f0c5bc"}
Sep 17 22:15:21.462901 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.462497    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:21.464064 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.464042    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" event={"ID":"fa2d3ca5-f2de-4673-a02f-54931db1ced4","Type":"ContainerStarted","Data":"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"}
Sep 17 22:15:21.464193 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.464078    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" event={"ID":"fa2d3ca5-f2de-4673-a02f-54931db1ced4","Type":"ContainerStarted","Data":"6acee398d5a7178c3a257cdf442ff0599f16d1a332d3513260c338e24680a364"}
Sep 17 22:15:21.464193 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.464152    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:21.483507 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.483457    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" podStartSLOduration=2.483443164 podStartE2EDuration="2.483443164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:15:21.480857495 +0000 UTC m=+2024.829591385" watchObservedRunningTime="2026-09-17 22:15:21.483443164 +0000 UTC m=+2024.832177044"
Sep 17 22:15:21.501102 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:21.501049    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" podStartSLOduration=2.501035057 podStartE2EDuration="2.501035057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:15:21.498150536 +0000 UTC m=+2024.846884407" watchObservedRunningTime="2026-09-17 22:15:21.501035057 +0000 UTC m=+2024.849768937"
Sep 17 22:15:32.470342 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:32.470308    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:15:32.470747 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:32.470385    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:15:54.741059 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.740967    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:15:54.745232 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.745208    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.752268 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.752246    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-813cee\""
Sep 17 22:15:54.761061 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.761036    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:15:54.781481 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.781450    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.781597 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.781490    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw5cc\" (UniqueName: \"kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.882560 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.882525    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.882560 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.882563    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw5cc\" (UniqueName: \"kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.883296 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.883261    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:54.894934 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:54.894912    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw5cc\" (UniqueName: \"kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc\") pod \"payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:55.055486 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.055398    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:55.338407 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.338379    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:15:55.340326 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:15:55.340299    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod388bfa92_21f3_4b89_92b3_cda0ed7e3e01.slice/crio-44586a93be04d65a766b2e90126c3178fad71fea61e44c836f7a97ea1af2330d WatchSource:0}: Error finding container 44586a93be04d65a766b2e90126c3178fad71fea61e44c836f7a97ea1af2330d: Status 404 returned error can't find the container with id 44586a93be04d65a766b2e90126c3178fad71fea61e44c836f7a97ea1af2330d
Sep 17 22:15:55.643744 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.643654    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" event={"ID":"388bfa92-21f3-4b89-92b3-cda0ed7e3e01","Type":"ContainerStarted","Data":"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"}
Sep 17 22:15:55.643744 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.643713    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" event={"ID":"388bfa92-21f3-4b89-92b3-cda0ed7e3e01","Type":"ContainerStarted","Data":"44586a93be04d65a766b2e90126c3178fad71fea61e44c836f7a97ea1af2330d"}
Sep 17 22:15:55.643956 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.643790    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:15:55.691001 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:15:55.690944    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" podStartSLOduration=1.690927518 podStartE2EDuration="1.690927518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:15:55.688503846 +0000 UTC m=+2059.037237731" watchObservedRunningTime="2026-09-17 22:15:55.690927518 +0000 UTC m=+2059.039661397"
Sep 17 22:16:06.651011 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:16:06.650976    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:16:26.337092 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:16:26.337059    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"
Sep 17 22:16:26.470121 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:16:26.470095    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp_388bfa92-21f3-4b89-92b3-cda0ed7e3e01/payload-processing/0.log"
Sep 17 22:16:38.961264 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:16:38.961235    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:16:38.964086 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:16:38.964067    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:17:02.755835 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:02.755799    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk_fa2d3ca5-f2de-4673-a02f-54931db1ced4/payload-processing/0.log"
Sep 17 22:17:02.878076 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:02.878044    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"
Sep 17 22:17:03.007808 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:03.007729    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp_388bfa92-21f3-4b89-92b3-cda0ed7e3e01/payload-processing/0.log"
Sep 17 22:17:03.143523 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:03.143495    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp_388bfa92-21f3-4b89-92b3-cda0ed7e3e01/payload-processing/0.log"
Sep 17 22:17:12.019449 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.019416    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:12.023112 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.023089    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.026097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.026077    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2e0396-serving-cert\""
Sep 17 22:17:12.040897 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.040866    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:12.041555 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.041528    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.041670 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.041620    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.041728 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.041713    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.041792 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.041773    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn548\" (UniqueName: \"kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.142740 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.142709    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.142912 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.142768    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.142912 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.142819    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.142912 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.142855    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn548\" (UniqueName: \"kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.143519 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.143493    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.145497 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.145470    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.145588 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.145543    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.153383 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.153359    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn548\" (UniqueName: \"kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548\") pod \"maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.333193 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.333103    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:12.619296 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.619244    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:12.621337 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:17:12.621310    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33431ca_b957_4ee9_bd77_5459f169398f.slice/crio-9e4a058b985d7b6ceee3d62ce12651d612229073a8834e6f0196676ffbd7aa55 WatchSource:0}: Error finding container 9e4a058b985d7b6ceee3d62ce12651d612229073a8834e6f0196676ffbd7aa55: Status 404 returned error can't find the container with id 9e4a058b985d7b6ceee3d62ce12651d612229073a8834e6f0196676ffbd7aa55
Sep 17 22:17:12.622708 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.622691    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 22:17:12.779104 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.779065    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:12.784020 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.784002    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.787208 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.787179    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-2e0396-dockercfg-l69b9\""
Sep 17 22:17:12.787382 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.787303    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-2e0396\""
Sep 17 22:17:12.800516 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.800488    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:12.848446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.848411    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrzh\" (UniqueName: \"kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.848446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.848451    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.949275 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.949200    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrzh\" (UniqueName: \"kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.949478 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.949446    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.949998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.949980    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:12.961021 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:12.960999    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrzh\" (UniqueName: \"kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh\") pod \"payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:13.076755 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.076705    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" event={"ID":"c33431ca-b957-4ee9-bd77-5459f169398f","Type":"ContainerStarted","Data":"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"}
Sep 17 22:17:13.077141 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.076804    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:13.077141 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.076830    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" event={"ID":"c33431ca-b957-4ee9-bd77-5459f169398f","Type":"ContainerStarted","Data":"9e4a058b985d7b6ceee3d62ce12651d612229073a8834e6f0196676ffbd7aa55"}
Sep 17 22:17:13.095863 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.095820    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:13.383355 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.383271    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" podStartSLOduration=2.037855751 podStartE2EDuration="2.383250877s" totalImagesPullingTime="345.395126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:17:13.135584824 +0000 UTC m=+2136.484318706" watchObservedRunningTime="2026-09-17 22:17:13.383250877 +0000 UTC m=+2136.731984756"
Sep 17 22:17:13.384432 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:13.384406    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:13.390691 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:17:13.390667    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6455eeb2_b26a_4aba_b820_e68985f672ff.slice/crio-49bf7d55fc0eebe6a66370551be53bfcedf31c579365b2014003ef6f7718dc12 WatchSource:0}: Error finding container 49bf7d55fc0eebe6a66370551be53bfcedf31c579365b2014003ef6f7718dc12: Status 404 returned error can't find the container with id 49bf7d55fc0eebe6a66370551be53bfcedf31c579365b2014003ef6f7718dc12
Sep 17 22:17:14.083587 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:14.083554    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" event={"ID":"6455eeb2-b26a-4aba-b820-e68985f672ff","Type":"ContainerStarted","Data":"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"}
Sep 17 22:17:14.084097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:14.083609    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" event={"ID":"6455eeb2-b26a-4aba-b820-e68985f672ff","Type":"ContainerStarted","Data":"49bf7d55fc0eebe6a66370551be53bfcedf31c579365b2014003ef6f7718dc12"}
Sep 17 22:17:14.108258 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:14.108199    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" podStartSLOduration=2.108182685 podStartE2EDuration="2.108182685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:17:14.105352611 +0000 UTC m=+2137.454086495" watchObservedRunningTime="2026-09-17 22:17:14.108182685 +0000 UTC m=+2137.456916605"
Sep 17 22:17:15.089521 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:15.089480    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:19.089633 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:19.089556    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:26.097308 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:26.097252    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:49.707510 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:49.707467    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:49.708051 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:49.707697    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" podUID="c33431ca-b957-4ee9-bd77-5459f169398f" containerName="maas-api" containerID="cri-o://684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4" gracePeriod=30
Sep 17 22:17:49.798075 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:49.798033    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:49.798318 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:49.798273    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" podUID="6455eeb2-b26a-4aba-b820-e68985f672ff" containerName="payload-pre-processing" containerID="cri-o://82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81" gracePeriod=30
Sep 17 22:17:49.983550 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:49.983523    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:50.073218 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.073194    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:50.104067 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.104031    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn548\" (UniqueName: \"kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548\") pod \"c33431ca-b957-4ee9-bd77-5459f169398f\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") "
Sep 17 22:17:50.104408 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.104389    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls\") pod \"c33431ca-b957-4ee9-bd77-5459f169398f\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") "
Sep 17 22:17:50.104549 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.104535    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca\") pod \"c33431ca-b957-4ee9-bd77-5459f169398f\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") "
Sep 17 22:17:50.104690 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.104676    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs\") pod \"c33431ca-b957-4ee9-bd77-5459f169398f\" (UID: \"c33431ca-b957-4ee9-bd77-5459f169398f\") "
Sep 17 22:17:50.107879 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.107827    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c33431ca-b957-4ee9-bd77-5459f169398f" (UID: "c33431ca-b957-4ee9-bd77-5459f169398f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:17:50.109657 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.109619    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c33431ca-b957-4ee9-bd77-5459f169398f" (UID: "c33431ca-b957-4ee9-bd77-5459f169398f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:17:50.109781 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.109747    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548" (OuterVolumeSpecName: "kube-api-access-zn548") pod "c33431ca-b957-4ee9-bd77-5459f169398f" (UID: "c33431ca-b957-4ee9-bd77-5459f169398f"). InnerVolumeSpecName "kube-api-access-zn548". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:17:50.110369 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.110350    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c33431ca-b957-4ee9-bd77-5459f169398f" (UID: "c33431ca-b957-4ee9-bd77-5459f169398f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:17:50.206305 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206256    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsrzh\" (UniqueName: \"kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh\") pod \"6455eeb2-b26a-4aba-b820-e68985f672ff\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") "
Sep 17 22:17:50.206474 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206408    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume\") pod \"6455eeb2-b26a-4aba-b820-e68985f672ff\" (UID: \"6455eeb2-b26a-4aba-b820-e68985f672ff\") "
Sep 17 22:17:50.206629 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206608    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c33431ca-b957-4ee9-bd77-5459f169398f-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.206668 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206631    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.206668 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206641    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn548\" (UniqueName: \"kubernetes.io/projected/c33431ca-b957-4ee9-bd77-5459f169398f-kube-api-access-zn548\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.206668 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206650    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c33431ca-b957-4ee9-bd77-5459f169398f-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.206817 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.206792    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6455eeb2-b26a-4aba-b820-e68985f672ff" (UID: "6455eeb2-b26a-4aba-b820-e68985f672ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:17:50.208603 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.208586    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh" (OuterVolumeSpecName: "kube-api-access-tsrzh") pod "6455eeb2-b26a-4aba-b820-e68985f672ff" (UID: "6455eeb2-b26a-4aba-b820-e68985f672ff"). InnerVolumeSpecName "kube-api-access-tsrzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:17:50.295097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.295061    2587 generic.go:356] "Generic (PLEG): container finished" podID="6455eeb2-b26a-4aba-b820-e68985f672ff" containerID="82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81" exitCode=0
Sep 17 22:17:50.295309 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.295117    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"
Sep 17 22:17:50.295309 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.295150    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" event={"ID":"6455eeb2-b26a-4aba-b820-e68985f672ff","Type":"ContainerDied","Data":"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"}
Sep 17 22:17:50.295309 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.295224    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r" event={"ID":"6455eeb2-b26a-4aba-b820-e68985f672ff","Type":"ContainerDied","Data":"49bf7d55fc0eebe6a66370551be53bfcedf31c579365b2014003ef6f7718dc12"}
Sep 17 22:17:50.295309 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.295254    2587 scope.go:117] "RemoveContainer" containerID="82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"
Sep 17 22:17:50.296610 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.296585    2587 generic.go:356] "Generic (PLEG): container finished" podID="c33431ca-b957-4ee9-bd77-5459f169398f" containerID="684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4" exitCode=0
Sep 17 22:17:50.296683 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.296615    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" event={"ID":"c33431ca-b957-4ee9-bd77-5459f169398f","Type":"ContainerDied","Data":"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"}
Sep 17 22:17:50.296683 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.296651    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"
Sep 17 22:17:50.296683 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.296666    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g" event={"ID":"c33431ca-b957-4ee9-bd77-5459f169398f","Type":"ContainerDied","Data":"9e4a058b985d7b6ceee3d62ce12651d612229073a8834e6f0196676ffbd7aa55"}
Sep 17 22:17:50.307342 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.307318    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6455eeb2-b26a-4aba-b820-e68985f672ff-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.307342 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.307343    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrzh\" (UniqueName: \"kubernetes.io/projected/6455eeb2-b26a-4aba-b820-e68985f672ff-kube-api-access-tsrzh\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:17:50.308115 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.308077    2587 scope.go:117] "RemoveContainer" containerID="82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"
Sep 17 22:17:50.308575 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:17:50.308452    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81\": container with ID starting with 82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81 not found: ID does not exist" containerID="82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"
Sep 17 22:17:50.308575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.308490    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81"} err="failed to get container status \"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81\": rpc error: code = NotFound desc = could not find container \"82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81\": container with ID starting with 82e6ebce7283c354e7f6e37e37cfe29c47ecdf1bd05c59bb604e1d0ab2863d81 not found: ID does not exist"
Sep 17 22:17:50.308575 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.308512    2587 scope.go:117] "RemoveContainer" containerID="684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"
Sep 17 22:17:50.317233 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.317214    2587 scope.go:117] "RemoveContainer" containerID="684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"
Sep 17 22:17:50.317655 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:17:50.317626    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4\": container with ID starting with 684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4 not found: ID does not exist" containerID="684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"
Sep 17 22:17:50.317735 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.317656    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4"} err="failed to get container status \"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4\": rpc error: code = NotFound desc = could not find container \"684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4\": container with ID starting with 684736e91444073d8461d200059416361e109e0d9192f00571499142feb636b4 not found: ID does not exist"
Sep 17 22:17:50.333518 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.333490    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:50.340415 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.340394    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2e0396-689fdbbcb9-rbt2g"]
Sep 17 22:17:50.368640 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.368614    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:50.389465 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.389437    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2e0396-7499b5c5c4-m4s8r"]
Sep 17 22:17:50.857633 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.857600    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6455eeb2-b26a-4aba-b820-e68985f672ff" path="/var/lib/kubelet/pods/6455eeb2-b26a-4aba-b820-e68985f672ff/volumes"
Sep 17 22:17:50.858009 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:17:50.857990    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c33431ca-b957-4ee9-bd77-5459f169398f" path="/var/lib/kubelet/pods/c33431ca-b957-4ee9-bd77-5459f169398f/volumes"
Sep 17 22:18:16.889115 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:16.889037    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:18:16.899524 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:16.899482    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:18:16.899765 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:16.899731    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" podUID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" containerName="payload-pre-processing" containerID="cri-o://face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1" gracePeriod=30
Sep 17 22:18:16.969390 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:16.969351    2587 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-ipp-a-a71836: configmap "payload-processing-plugins-e2e-ipp-a-a71836" not found
Sep 17 22:18:16.969580 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:16.969457    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume podName:fa2d3ca5-f2de-4673-a02f-54931db1ced4 nodeName:}" failed. No retries permitted until 2026-09-17 22:18:17.469433614 +0000 UTC m=+2200.818167495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume") pod "payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" (UID: "fa2d3ca5-f2de-4673-a02f-54931db1ced4") : configmap "payload-processing-plugins-e2e-ipp-a-a71836" not found
Sep 17 22:18:17.159799 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.159758    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:18:17.272846 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.272805    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ct7z\" (UniqueName: \"kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z\") pod \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") "
Sep 17 22:18:17.272846 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.272847    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume\") pod \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\" (UID: \"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c\") "
Sep 17 22:18:17.273354 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.273274    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" (UID: "efaaff4f-50a6-49a7-b146-17c6d8c9cf5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:18:17.275176 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.275155    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z" (OuterVolumeSpecName: "kube-api-access-5ct7z") pod "efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" (UID: "efaaff4f-50a6-49a7-b146-17c6d8c9cf5c"). InnerVolumeSpecName "kube-api-access-5ct7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:18:17.374535 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.374497    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ct7z\" (UniqueName: \"kubernetes.io/projected/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-kube-api-access-5ct7z\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:18:17.374535 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.374527    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:18:17.441559 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441460    2587 generic.go:356] "Generic (PLEG): container finished" podID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" containerID="face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1" exitCode=0
Sep 17 22:18:17.441559 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441498    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"
Sep 17 22:18:17.441559 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441546    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" event={"ID":"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c","Type":"ContainerDied","Data":"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"}
Sep 17 22:18:17.441826 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441599    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd" event={"ID":"efaaff4f-50a6-49a7-b146-17c6d8c9cf5c","Type":"ContainerDied","Data":"e0136a9636ee8b66b4d79402181b1589e43e85831e9f519a6cb66237d2f0c5bc"}
Sep 17 22:18:17.441826 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441671    2587 scope.go:117] "RemoveContainer" containerID="face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"
Sep 17 22:18:17.441826 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.441719    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" podUID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" containerName="payload-processing" containerID="cri-o://06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35" gracePeriod=30
Sep 17 22:18:17.452087 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.452065    2587 scope.go:117] "RemoveContainer" containerID="face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"
Sep 17 22:18:17.452465 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:17.452444    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1\": container with ID starting with face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1 not found: ID does not exist" containerID="face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"
Sep 17 22:18:17.452544 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.452475    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1"} err="failed to get container status \"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1\": rpc error: code = NotFound desc = could not find container \"face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1\": container with ID starting with face4653a4e99aeae735f721e6b4f6af134c1db4b6d2c50f1b97ba7fb153ecc1 not found: ID does not exist"
Sep 17 22:18:17.475966 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:17.475935    2587 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-ipp-a-a71836: configmap "payload-processing-plugins-e2e-ipp-a-a71836" not found
Sep 17 22:18:17.476094 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:17.476039    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume podName:fa2d3ca5-f2de-4673-a02f-54931db1ced4 nodeName:}" failed. No retries permitted until 2026-09-17 22:18:18.47601729 +0000 UTC m=+2201.824751168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume") pod "payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" (UID: "fa2d3ca5-f2de-4673-a02f-54931db1ced4") : configmap "payload-processing-plugins-e2e-ipp-a-a71836" not found
Sep 17 22:18:17.523118 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.523081    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:18:17.529800 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.529771    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a71836-f785687fd-s6qvd"]
Sep 17 22:18:17.688111 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.688084    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:18:17.779057 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.779024    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume\") pod \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") "
Sep 17 22:18:17.779241 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.779174    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58psh\" (UniqueName: \"kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh\") pod \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\" (UID: \"fa2d3ca5-f2de-4673-a02f-54931db1ced4\") "
Sep 17 22:18:17.779434 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.779411    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa2d3ca5-f2de-4673-a02f-54931db1ced4" (UID: "fa2d3ca5-f2de-4673-a02f-54931db1ced4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:18:17.781469 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.781445    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh" (OuterVolumeSpecName: "kube-api-access-58psh") pod "fa2d3ca5-f2de-4673-a02f-54931db1ced4" (UID: "fa2d3ca5-f2de-4673-a02f-54931db1ced4"). InnerVolumeSpecName "kube-api-access-58psh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:18:17.879948 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.879910    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa2d3ca5-f2de-4673-a02f-54931db1ced4-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:18:17.879948 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:17.879939    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58psh\" (UniqueName: \"kubernetes.io/projected/fa2d3ca5-f2de-4673-a02f-54931db1ced4-kube-api-access-58psh\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:18:18.448082 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.448045    2587 generic.go:356] "Generic (PLEG): container finished" podID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" containerID="06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35" exitCode=0
Sep 17 22:18:18.448595 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.448087    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"
Sep 17 22:18:18.448595 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.448140    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" event={"ID":"fa2d3ca5-f2de-4673-a02f-54931db1ced4","Type":"ContainerDied","Data":"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"}
Sep 17 22:18:18.448595 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.448190    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk" event={"ID":"fa2d3ca5-f2de-4673-a02f-54931db1ced4","Type":"ContainerDied","Data":"6acee398d5a7178c3a257cdf442ff0599f16d1a332d3513260c338e24680a364"}
Sep 17 22:18:18.448595 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.448220    2587 scope.go:117] "RemoveContainer" containerID="06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"
Sep 17 22:18:18.460355 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.460340    2587 scope.go:117] "RemoveContainer" containerID="06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"
Sep 17 22:18:18.460863 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:18:18.460842    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35\": container with ID starting with 06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35 not found: ID does not exist" containerID="06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"
Sep 17 22:18:18.460940 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.460870    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35"} err="failed to get container status \"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35\": rpc error: code = NotFound desc = could not find container \"06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35\": container with ID starting with 06ede52380ac8bbc7e79e1e701c0f480c3f50130ff33e852d98e591733ed7b35 not found: ID does not exist"
Sep 17 22:18:18.478582 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.478553    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:18:18.501120 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.501091    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a71836-7f85579b5c-2k2sk"]
Sep 17 22:18:18.857299 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.857259    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" path="/var/lib/kubelet/pods/efaaff4f-50a6-49a7-b146-17c6d8c9cf5c/volumes"
Sep 17 22:18:18.857670 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:18:18.857656    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" path="/var/lib/kubelet/pods/fa2d3ca5-f2de-4673-a02f-54931db1ced4/volumes"
Sep 17 22:19:33.095592 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.095558    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:19:33.096154 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.095826    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" podUID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" containerName="payload-processing" containerID="cri-o://6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b" gracePeriod=30
Sep 17 22:19:33.356745 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.356671    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:19:33.442070 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.442028    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume\") pod \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") "
Sep 17 22:19:33.442070 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.442072    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw5cc\" (UniqueName: \"kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc\") pod \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\" (UID: \"388bfa92-21f3-4b89-92b3-cda0ed7e3e01\") "
Sep 17 22:19:33.442539 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.442507    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "388bfa92-21f3-4b89-92b3-cda0ed7e3e01" (UID: "388bfa92-21f3-4b89-92b3-cda0ed7e3e01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:19:33.444570 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.444540    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc" (OuterVolumeSpecName: "kube-api-access-pw5cc") pod "388bfa92-21f3-4b89-92b3-cda0ed7e3e01" (UID: "388bfa92-21f3-4b89-92b3-cda0ed7e3e01"). InnerVolumeSpecName "kube-api-access-pw5cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:19:33.542833 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.542786    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:19:33.542833 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.542819    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw5cc\" (UniqueName: \"kubernetes.io/projected/388bfa92-21f3-4b89-92b3-cda0ed7e3e01-kube-api-access-pw5cc\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:19:33.856574 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.856537    2587 generic.go:356] "Generic (PLEG): container finished" podID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" containerID="6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b" exitCode=0
Sep 17 22:19:33.856748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.856590    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" event={"ID":"388bfa92-21f3-4b89-92b3-cda0ed7e3e01","Type":"ContainerDied","Data":"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"}
Sep 17 22:19:33.856748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.856631    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp" event={"ID":"388bfa92-21f3-4b89-92b3-cda0ed7e3e01","Type":"ContainerDied","Data":"44586a93be04d65a766b2e90126c3178fad71fea61e44c836f7a97ea1af2330d"}
Sep 17 22:19:33.856748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.856634    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"
Sep 17 22:19:33.856748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.856651    2587 scope.go:117] "RemoveContainer" containerID="6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"
Sep 17 22:19:33.866108 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.866091    2587 scope.go:117] "RemoveContainer" containerID="6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"
Sep 17 22:19:33.866500 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:19:33.866477    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b\": container with ID starting with 6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b not found: ID does not exist" containerID="6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"
Sep 17 22:19:33.866592 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.866506    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b"} err="failed to get container status \"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b\": rpc error: code = NotFound desc = could not find container \"6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b\": container with ID starting with 6f940fb18c6b4daaa410fc71d8fa7184b73db9b2602845522cf821673e36bf4b not found: ID does not exist"
Sep 17 22:19:33.887768 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.887740    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:19:33.895642 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:33.895614    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-813cee-6fd6bb5cb5-9pgpp"]
Sep 17 22:19:34.858241 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:34.858205    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" path="/var/lib/kubelet/pods/388bfa92-21f3-4b89-92b3-cda0ed7e3e01/volumes"
Sep 17 22:19:51.118339 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.118303    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:19:51.118856 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.118553    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" podUID="b698c377-c199-4fe3-b28a-4ce61424631f" containerName="payload-processing" containerID="cri-o://7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4" gracePeriod=30
Sep 17 22:19:51.385975 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.385950    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:19:51.413626 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.413595    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qj8b\" (UniqueName: \"kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b\") pod \"b698c377-c199-4fe3-b28a-4ce61424631f\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") "
Sep 17 22:19:51.413802 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.413680    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume\") pod \"b698c377-c199-4fe3-b28a-4ce61424631f\" (UID: \"b698c377-c199-4fe3-b28a-4ce61424631f\") "
Sep 17 22:19:51.414073 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.414041    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b698c377-c199-4fe3-b28a-4ce61424631f" (UID: "b698c377-c199-4fe3-b28a-4ce61424631f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:19:51.416615 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.416587    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b" (OuterVolumeSpecName: "kube-api-access-6qj8b") pod "b698c377-c199-4fe3-b28a-4ce61424631f" (UID: "b698c377-c199-4fe3-b28a-4ce61424631f"). InnerVolumeSpecName "kube-api-access-6qj8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:19:51.515236 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.515189    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qj8b\" (UniqueName: \"kubernetes.io/projected/b698c377-c199-4fe3-b28a-4ce61424631f-kube-api-access-6qj8b\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:19:51.515236 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.515223    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b698c377-c199-4fe3-b28a-4ce61424631f-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:19:51.952681 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.952648    2587 generic.go:356] "Generic (PLEG): container finished" podID="b698c377-c199-4fe3-b28a-4ce61424631f" containerID="7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4" exitCode=0
Sep 17 22:19:51.952870 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.952694    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"
Sep 17 22:19:51.952870 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.952733    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" event={"ID":"b698c377-c199-4fe3-b28a-4ce61424631f","Type":"ContainerDied","Data":"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"}
Sep 17 22:19:51.952870 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.952782    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w" event={"ID":"b698c377-c199-4fe3-b28a-4ce61424631f","Type":"ContainerDied","Data":"bd410d65ca1948d78af018de4b5249247f66d649b077f5572cec7f6c168cc184"}
Sep 17 22:19:51.952870 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.952803    2587 scope.go:117] "RemoveContainer" containerID="7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"
Sep 17 22:19:51.962335 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.962313    2587 scope.go:117] "RemoveContainer" containerID="7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"
Sep 17 22:19:51.962605 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:19:51.962584    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4\": container with ID starting with 7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4 not found: ID does not exist" containerID="7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"
Sep 17 22:19:51.962677 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.962625    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4"} err="failed to get container status \"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4\": rpc error: code = NotFound desc = could not find container \"7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4\": container with ID starting with 7c135e00bc98ccc7f47e82a8b8f590c4630fd6479feb2aae8b683f197f2385a4 not found: ID does not exist"
Sep 17 22:19:51.977722 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.977691    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:19:51.979609 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:51.979588    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8f8dad-54dd945d77-n2r2w"]
Sep 17 22:19:52.857754 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:19:52.857721    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b698c377-c199-4fe3-b28a-4ce61424631f" path="/var/lib/kubelet/pods/b698c377-c199-4fe3-b28a-4ce61424631f/volumes"
Sep 17 22:20:09.254425 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.254389    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:20:09.254903 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.254854    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" podUID="b2d172c6-b413-4183-a516-04d710e2a537" containerName="payload-pre-processing" containerID="cri-o://3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196" gracePeriod=30
Sep 17 22:20:09.518712 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.518652    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:20:09.601982 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.601949    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf8h9\" (UniqueName: \"kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9\") pod \"b2d172c6-b413-4183-a516-04d710e2a537\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") "
Sep 17 22:20:09.602154 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.602058    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume\") pod \"b2d172c6-b413-4183-a516-04d710e2a537\" (UID: \"b2d172c6-b413-4183-a516-04d710e2a537\") "
Sep 17 22:20:09.602450 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.602426    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2d172c6-b413-4183-a516-04d710e2a537" (UID: "b2d172c6-b413-4183-a516-04d710e2a537"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:20:09.604220 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.604192    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9" (OuterVolumeSpecName: "kube-api-access-lf8h9") pod "b2d172c6-b413-4183-a516-04d710e2a537" (UID: "b2d172c6-b413-4183-a516-04d710e2a537"). InnerVolumeSpecName "kube-api-access-lf8h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:20:09.703641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.703598    2587 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d172c6-b413-4183-a516-04d710e2a537-plugins-config-volume\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:20:09.703641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:09.703634    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf8h9\" (UniqueName: \"kubernetes.io/projected/b2d172c6-b413-4183-a516-04d710e2a537-kube-api-access-lf8h9\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:20:10.048500 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.048468    2587 generic.go:356] "Generic (PLEG): container finished" podID="b2d172c6-b413-4183-a516-04d710e2a537" containerID="3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196" exitCode=0
Sep 17 22:20:10.048762 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.048515    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"
Sep 17 22:20:10.048762 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.048533    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" event={"ID":"b2d172c6-b413-4183-a516-04d710e2a537","Type":"ContainerDied","Data":"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"}
Sep 17 22:20:10.048762 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.048577    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk" event={"ID":"b2d172c6-b413-4183-a516-04d710e2a537","Type":"ContainerDied","Data":"c3f09360dc91c8d19e0d07324175fafcfbcd3770e49c16bb4ad79b9a5c6e2671"}
Sep 17 22:20:10.048762 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.048598    2587 scope.go:117] "RemoveContainer" containerID="3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"
Sep 17 22:20:10.057679 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.057657    2587 scope.go:117] "RemoveContainer" containerID="3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"
Sep 17 22:20:10.057971 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:20:10.057952    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196\": container with ID starting with 3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196 not found: ID does not exist" containerID="3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"
Sep 17 22:20:10.058015 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.057981    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196"} err="failed to get container status \"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196\": rpc error: code = NotFound desc = could not find container \"3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196\": container with ID starting with 3a875d4637f53e2b86bbd12cfa41f3e3c60d49528f92d0d67f2fc263dbb4c196 not found: ID does not exist"
Sep 17 22:20:10.073736 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.073689    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:20:10.076135 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.076112    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-35b181-769fc5d698-928hk"]
Sep 17 22:20:10.857442 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:10.857404    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d172c6-b413-4183-a516-04d710e2a537" path="/var/lib/kubelet/pods/b2d172c6-b413-4183-a516-04d710e2a537/volumes"
Sep 17 22:20:19.655854 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:19.655759    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 22:20:19.656333 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:19.656055    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b978497bd-l7kd7" podUID="13488348-10bf-41c5-8aed-102668246b5d" containerName="manager" containerID="cri-o://eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635" gracePeriod=30
Sep 17 22:20:19.908119 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:19.908060    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 22:20:20.099624 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.099593    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert\") pod \"13488348-10bf-41c5-8aed-102668246b5d\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") "
Sep 17 22:20:20.099833 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.099653    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs\") pod \"13488348-10bf-41c5-8aed-102668246b5d\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") "
Sep 17 22:20:20.099833 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.099692    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw42h\" (UniqueName: \"kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h\") pod \"13488348-10bf-41c5-8aed-102668246b5d\" (UID: \"13488348-10bf-41c5-8aed-102668246b5d\") "
Sep 17 22:20:20.101087 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.101054    2587 generic.go:356] "Generic (PLEG): container finished" podID="13488348-10bf-41c5-8aed-102668246b5d" containerID="eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635" exitCode=0
Sep 17 22:20:20.101228 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.101172    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-l7kd7"
Sep 17 22:20:20.101228 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.101200    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-l7kd7" event={"ID":"13488348-10bf-41c5-8aed-102668246b5d","Type":"ContainerDied","Data":"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"}
Sep 17 22:20:20.101420 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.101268    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-l7kd7" event={"ID":"13488348-10bf-41c5-8aed-102668246b5d","Type":"ContainerDied","Data":"01a7e9511afef820361517ad4cff97484deb78db8238f4934d91e127d6fa1707"}
Sep 17 22:20:20.101420 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.101323    2587 scope.go:117] "RemoveContainer" containerID="eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"
Sep 17 22:20:20.102212 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.102181    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h" (OuterVolumeSpecName: "kube-api-access-rw42h") pod "13488348-10bf-41c5-8aed-102668246b5d" (UID: "13488348-10bf-41c5-8aed-102668246b5d"). InnerVolumeSpecName "kube-api-access-rw42h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:20:20.102331 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.102188    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "13488348-10bf-41c5-8aed-102668246b5d" (UID: "13488348-10bf-41c5-8aed-102668246b5d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:20:20.102331 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.102272    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "13488348-10bf-41c5-8aed-102668246b5d" (UID: "13488348-10bf-41c5-8aed-102668246b5d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:20:20.121201 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.121179    2587 scope.go:117] "RemoveContainer" containerID="eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"
Sep 17 22:20:20.121527 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:20:20.121504    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635\": container with ID starting with eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635 not found: ID does not exist" containerID="eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"
Sep 17 22:20:20.121625 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.121534    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635"} err="failed to get container status \"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635\": rpc error: code = NotFound desc = could not find container \"eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635\": container with ID starting with eda1697365c975c8515ef1723dca018a53d918e53bc21663add38715c6657635 not found: ID does not exist"
Sep 17 22:20:20.201527 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.201447    2587 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-webhook-cert\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:20:20.201527 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.201475    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13488348-10bf-41c5-8aed-102668246b5d-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:20:20.201527 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.201484    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw42h\" (UniqueName: \"kubernetes.io/projected/13488348-10bf-41c5-8aed-102668246b5d-kube-api-access-rw42h\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:20:20.426420 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.426387    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 22:20:20.433226 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.433194    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b978497bd-l7kd7"]
Sep 17 22:20:20.857410 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:20:20.857376    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13488348-10bf-41c5-8aed-102668246b5d" path="/var/lib/kubelet/pods/13488348-10bf-41c5-8aed-102668246b5d/volumes"
Sep 17 22:21:21.278660 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.278621    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279009    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c33431ca-b957-4ee9-bd77-5459f169398f" containerName="maas-api"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279021    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="c33431ca-b957-4ee9-bd77-5459f169398f" containerName="maas-api"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279028    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d172c6-b413-4183-a516-04d710e2a537" containerName="payload-pre-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279034    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d172c6-b413-4183-a516-04d710e2a537" containerName="payload-pre-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279049    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279054    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279062    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279067    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279082    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b698c377-c199-4fe3-b28a-4ce61424631f" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279087    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b698c377-c199-4fe3-b28a-4ce61424631f" containerName="payload-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279099    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6455eeb2-b26a-4aba-b820-e68985f672ff" containerName="payload-pre-processing"
Sep 17 22:21:21.279097 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279104    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="6455eeb2-b26a-4aba-b820-e68985f672ff" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279113    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279118    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279126    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13488348-10bf-41c5-8aed-102668246b5d" containerName="manager"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279132    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="13488348-10bf-41c5-8aed-102668246b5d" containerName="manager"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279183    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="efaaff4f-50a6-49a7-b146-17c6d8c9cf5c" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279192    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="13488348-10bf-41c5-8aed-102668246b5d" containerName="manager"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279198    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="c33431ca-b957-4ee9-bd77-5459f169398f" containerName="maas-api"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279203    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d172c6-b413-4183-a516-04d710e2a537" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279209    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="388bfa92-21f3-4b89-92b3-cda0ed7e3e01" containerName="payload-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279215    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="6455eeb2-b26a-4aba-b820-e68985f672ff" containerName="payload-pre-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279221    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa2d3ca5-f2de-4673-a02f-54931db1ced4" containerName="payload-processing"
Sep 17 22:21:21.279502 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.279226    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b698c377-c199-4fe3-b28a-4ce61424631f" containerName="payload-processing"
Sep 17 22:21:21.282263 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.282245    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.286109 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.286070    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 22:21:21.287540 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.287518    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9wpkz\""
Sep 17 22:21:21.287659 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.287543    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 22:21:21.287659 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.287554    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 22:21:21.302335 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.302306    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:21.378641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.378597    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.378641 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.378645    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.378849 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.378720    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jsjz\" (UniqueName: \"kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.479686 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.479650    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.479863 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.479695    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.479863 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.479750    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jsjz\" (UniqueName: \"kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.482371 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.482346    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.482501 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.482479    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.489227 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.489200    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jsjz\" (UniqueName: \"kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz\") pod \"maas-controller-b978497bd-vj8zj\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") " pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.593180 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.593077    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:21.869014 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:21.868990    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:21.871813 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:21:21.871781    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08563779_bb2e_4ff3_870e_32b875a1c453.slice/crio-b1b431eff8fdc31a8ab510628139ef9192dc25dd8d9a4b60d2bff71dd32bfb57 WatchSource:0}: Error finding container b1b431eff8fdc31a8ab510628139ef9192dc25dd8d9a4b60d2bff71dd32bfb57: Status 404 returned error can't find the container with id b1b431eff8fdc31a8ab510628139ef9192dc25dd8d9a4b60d2bff71dd32bfb57
Sep 17 22:21:22.409304 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:22.409253    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-vj8zj" event={"ID":"08563779-bb2e-4ff3-870e-32b875a1c453","Type":"ContainerStarted","Data":"b1b431eff8fdc31a8ab510628139ef9192dc25dd8d9a4b60d2bff71dd32bfb57"}
Sep 17 22:21:23.415623 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:23.415589    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-vj8zj" event={"ID":"08563779-bb2e-4ff3-870e-32b875a1c453","Type":"ContainerStarted","Data":"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"}
Sep 17 22:21:23.416004 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:23.415721    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:23.435234 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:23.435183    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b978497bd-vj8zj" podStartSLOduration=1.852364738 podStartE2EDuration="2.435169513s" totalImagesPullingTime="582.804775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:21:23.43324742 +0000 UTC m=+2386.781981326" watchObservedRunningTime="2026-09-17 22:21:23.435169513 +0000 UTC m=+2386.783903461"
Sep 17 22:21:34.423335 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:34.423303    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:38.994745 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:38.994713    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:21:38.999879 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:38.999858    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:21:43.804949 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:43.804911    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:43.805401 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:43.805150    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b978497bd-vj8zj" podUID="08563779-bb2e-4ff3-870e-32b875a1c453" containerName="manager" containerID="cri-o://ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165" gracePeriod=30
Sep 17 22:21:44.055915 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.055847    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:44.102807 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.102762    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert\") pod \"08563779-bb2e-4ff3-870e-32b875a1c453\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") "
Sep 17 22:21:44.102972 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.102902    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jsjz\" (UniqueName: \"kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz\") pod \"08563779-bb2e-4ff3-870e-32b875a1c453\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") "
Sep 17 22:21:44.102972 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.102946    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs\") pod \"08563779-bb2e-4ff3-870e-32b875a1c453\" (UID: \"08563779-bb2e-4ff3-870e-32b875a1c453\") "
Sep 17 22:21:44.105151 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.105116    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "08563779-bb2e-4ff3-870e-32b875a1c453" (UID: "08563779-bb2e-4ff3-870e-32b875a1c453"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:21:44.105273 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.105181    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz" (OuterVolumeSpecName: "kube-api-access-7jsjz") pod "08563779-bb2e-4ff3-870e-32b875a1c453" (UID: "08563779-bb2e-4ff3-870e-32b875a1c453"). InnerVolumeSpecName "kube-api-access-7jsjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:21:44.105273 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.105250    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "08563779-bb2e-4ff3-870e-32b875a1c453" (UID: "08563779-bb2e-4ff3-870e-32b875a1c453"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:21:44.204451 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.204415    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:21:44.204451 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.204446    2587 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08563779-bb2e-4ff3-870e-32b875a1c453-webhook-cert\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:21:44.204451 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.204455    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jsjz\" (UniqueName: \"kubernetes.io/projected/08563779-bb2e-4ff3-870e-32b875a1c453-kube-api-access-7jsjz\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:21:44.519601 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.519564    2587 generic.go:356] "Generic (PLEG): container finished" podID="08563779-bb2e-4ff3-870e-32b875a1c453" containerID="ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165" exitCode=0
Sep 17 22:21:44.519804 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.519622    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-vj8zj"
Sep 17 22:21:44.519804 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.519648    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-vj8zj" event={"ID":"08563779-bb2e-4ff3-870e-32b875a1c453","Type":"ContainerDied","Data":"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"}
Sep 17 22:21:44.519804 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.519705    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-vj8zj" event={"ID":"08563779-bb2e-4ff3-870e-32b875a1c453","Type":"ContainerDied","Data":"b1b431eff8fdc31a8ab510628139ef9192dc25dd8d9a4b60d2bff71dd32bfb57"}
Sep 17 22:21:44.519804 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.519726    2587 scope.go:117] "RemoveContainer" containerID="ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"
Sep 17 22:21:44.529265 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.529247    2587 scope.go:117] "RemoveContainer" containerID="ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"
Sep 17 22:21:44.529573 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:21:44.529554    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165\": container with ID starting with ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165 not found: ID does not exist" containerID="ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"
Sep 17 22:21:44.529627 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.529584    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165"} err="failed to get container status \"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165\": rpc error: code = NotFound desc = could not find container \"ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165\": container with ID starting with ec7c7d9c6b2ef1c0c05c5ae4820b9351c48722833de6072ae32c69ed092b3165 not found: ID does not exist"
Sep 17 22:21:44.542481 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.542453    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:44.546159 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.546140    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b978497bd-vj8zj"]
Sep 17 22:21:44.857380 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:44.857273    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08563779-bb2e-4ff3-870e-32b875a1c453" path="/var/lib/kubelet/pods/08563779-bb2e-4ff3-870e-32b875a1c453/volumes"
Sep 17 22:21:45.399714 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.399676    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b978497bd-sncsz"]
Sep 17 22:21:45.400085 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.400073    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08563779-bb2e-4ff3-870e-32b875a1c453" containerName="manager"
Sep 17 22:21:45.400136 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.400087    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="08563779-bb2e-4ff3-870e-32b875a1c453" containerName="manager"
Sep 17 22:21:45.400171 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.400147    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="08563779-bb2e-4ff3-870e-32b875a1c453" containerName="manager"
Sep 17 22:21:45.404582 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.404566    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.411225 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.410994    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9wpkz\""
Sep 17 22:21:45.412490 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.412467    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 22:21:45.412590 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.412515    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 22:21:45.412590 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.412556    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 22:21:45.435787 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.435764    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-sncsz"]
Sep 17 22:21:45.517814 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.517780    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-webhook-cert\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.517998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.517832    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-metrics-certs\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.517998 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.517854    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2d9n\" (UniqueName: \"kubernetes.io/projected/aed3d9d8-4510-41e7-b587-a48175875141-kube-api-access-m2d9n\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.619214 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.619178    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-webhook-cert\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.619445 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.619230    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-metrics-certs\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.619445 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.619249    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2d9n\" (UniqueName: \"kubernetes.io/projected/aed3d9d8-4510-41e7-b587-a48175875141-kube-api-access-m2d9n\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.621845 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.621821    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-webhook-cert\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.621943 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.621903    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed3d9d8-4510-41e7-b587-a48175875141-metrics-certs\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.628570 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.628541    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2d9n\" (UniqueName: \"kubernetes.io/projected/aed3d9d8-4510-41e7-b587-a48175875141-kube-api-access-m2d9n\") pod \"maas-controller-b978497bd-sncsz\" (UID: \"aed3d9d8-4510-41e7-b587-a48175875141\") " pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.714810 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.714704    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:45.989576 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:45.989552    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b978497bd-sncsz"]
Sep 17 22:21:45.991851 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:21:45.991823    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed3d9d8_4510_41e7_b587_a48175875141.slice/crio-50ec3bcdc97b94e89907d069ec62503f065e43f97f34437fcd3840b5797e1b31 WatchSource:0}: Error finding container 50ec3bcdc97b94e89907d069ec62503f065e43f97f34437fcd3840b5797e1b31: Status 404 returned error can't find the container with id 50ec3bcdc97b94e89907d069ec62503f065e43f97f34437fcd3840b5797e1b31
Sep 17 22:21:46.532856 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:46.532818    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-sncsz" event={"ID":"aed3d9d8-4510-41e7-b587-a48175875141","Type":"ContainerStarted","Data":"504cfad620581ea90c13f26b4247a8c9fa5ba7772345726d25bcfcea615965b9"}
Sep 17 22:21:46.533037 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:46.532888    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:21:46.533037 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:46.532915    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b978497bd-sncsz" event={"ID":"aed3d9d8-4510-41e7-b587-a48175875141","Type":"ContainerStarted","Data":"50ec3bcdc97b94e89907d069ec62503f065e43f97f34437fcd3840b5797e1b31"}
Sep 17 22:21:46.555865 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:46.555809    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b978497bd-sncsz" podStartSLOduration=1.099018901 podStartE2EDuration="1.555791208s" totalImagesPullingTime="456.772307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:21:46.553185713 +0000 UTC m=+2409.901919594" watchObservedRunningTime="2026-09-17 22:21:46.555791208 +0000 UTC m=+2409.904525088"
Sep 17 22:21:57.541032 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:21:57.540948    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b978497bd-sncsz"
Sep 17 22:25:57.456742 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.456705    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"]
Sep 17 22:25:57.460652 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.460628    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.474008 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.473983    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"]
Sep 17 22:25:57.541834 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.541796    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d217209-36d7-4eb7-ac32-18025f84e247-openshift-service-ca\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.541834 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.541837    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8h4c\" (UniqueName: \"kubernetes.io/projected/1d217209-36d7-4eb7-ac32-18025f84e247-kube-api-access-v8h4c\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.542060 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.541941    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-maas-api-tls\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.542060 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.541969    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-metrics-certs\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.643038 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.642993    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-maas-api-tls\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.643038 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.643034    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-metrics-certs\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.643328 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.643109    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d217209-36d7-4eb7-ac32-18025f84e247-openshift-service-ca\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.643328 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.643128    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8h4c\" (UniqueName: \"kubernetes.io/projected/1d217209-36d7-4eb7-ac32-18025f84e247-kube-api-access-v8h4c\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.643893 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.643810    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d217209-36d7-4eb7-ac32-18025f84e247-openshift-service-ca\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.645660 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.645640    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-metrics-certs\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.645748 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.645691    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d217209-36d7-4eb7-ac32-18025f84e247-maas-api-tls\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.653405 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.653381    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8h4c\" (UniqueName: \"kubernetes.io/projected/1d217209-36d7-4eb7-ac32-18025f84e247-kube-api-access-v8h4c\") pod \"maas-api-c966cd48f-jxh7l\" (UID: \"1d217209-36d7-4eb7-ac32-18025f84e247\") " pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:57.772259 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:57.772214    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:58.046606 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.046579    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"]
Sep 17 22:25:58.049076 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:25:58.049053    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d217209_36d7_4eb7_ac32_18025f84e247.slice/crio-bddd231b1e5903e85a30e49a2d4289bec113b40b9b7798410c9fa1d740c81929 WatchSource:0}: Error finding container bddd231b1e5903e85a30e49a2d4289bec113b40b9b7798410c9fa1d740c81929: Status 404 returned error can't find the container with id bddd231b1e5903e85a30e49a2d4289bec113b40b9b7798410c9fa1d740c81929
Sep 17 22:25:58.050809 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.050791    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 22:25:58.844189 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.844150    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l" event={"ID":"1d217209-36d7-4eb7-ac32-18025f84e247","Type":"ContainerStarted","Data":"bb965067d8184c29bd54e1659d8dbecfad30a2c49243b614ffd8bd1ad3f73e44"}
Sep 17 22:25:58.844636 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.844206    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l" event={"ID":"1d217209-36d7-4eb7-ac32-18025f84e247","Type":"ContainerStarted","Data":"bddd231b1e5903e85a30e49a2d4289bec113b40b9b7798410c9fa1d740c81929"}
Sep 17 22:25:58.844636 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.844275    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:25:58.867645 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:25:58.867588    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l" podStartSLOduration=1.539896567 podStartE2EDuration="1.867572207s" totalImagesPullingTime="327.67564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:25:58.864772411 +0000 UTC m=+2662.213506291" watchObservedRunningTime="2026-09-17 22:25:58.867572207 +0000 UTC m=+2662.216306152"
Sep 17 22:26:04.858628 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:04.858594    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c966cd48f-jxh7l"
Sep 17 22:26:04.914329 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:04.914296    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 22:26:04.914513 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:04.914484    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" podUID="3b29350a-c3ee-464e-aa6f-c250c68bd756" containerName="maas-api" containerID="cri-o://1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd" gracePeriod=30
Sep 17 22:26:06.164903 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.164877    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 22:26:06.223392 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.223364    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs\") pod \"3b29350a-c3ee-464e-aa6f-c250c68bd756\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") "
Sep 17 22:26:06.223554 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.223446    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") pod \"3b29350a-c3ee-464e-aa6f-c250c68bd756\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") "
Sep 17 22:26:06.223554 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.223468    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca\") pod \"3b29350a-c3ee-464e-aa6f-c250c68bd756\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") "
Sep 17 22:26:06.223554 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.223514    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf8sg\" (UniqueName: \"kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg\") pod \"3b29350a-c3ee-464e-aa6f-c250c68bd756\" (UID: \"3b29350a-c3ee-464e-aa6f-c250c68bd756\") "
Sep 17 22:26:06.223883 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.223845    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b29350a-c3ee-464e-aa6f-c250c68bd756" (UID: "3b29350a-c3ee-464e-aa6f-c250c68bd756"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:26:06.225616 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.225593    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3b29350a-c3ee-464e-aa6f-c250c68bd756" (UID: "3b29350a-c3ee-464e-aa6f-c250c68bd756"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:26:06.225747 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.225719    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg" (OuterVolumeSpecName: "kube-api-access-rf8sg") pod "3b29350a-c3ee-464e-aa6f-c250c68bd756" (UID: "3b29350a-c3ee-464e-aa6f-c250c68bd756"). InnerVolumeSpecName "kube-api-access-rf8sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:26:06.225878 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.225858    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b29350a-c3ee-464e-aa6f-c250c68bd756" (UID: "3b29350a-c3ee-464e-aa6f-c250c68bd756"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:26:06.324573 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.324519    2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-maas-api-tls\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:26:06.324573 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.324567    2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b29350a-c3ee-464e-aa6f-c250c68bd756-openshift-service-ca\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:26:06.324573 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.324578    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf8sg\" (UniqueName: \"kubernetes.io/projected/3b29350a-c3ee-464e-aa6f-c250c68bd756-kube-api-access-rf8sg\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:26:06.324573 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.324586    2587 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b29350a-c3ee-464e-aa6f-c250c68bd756-metrics-certs\") on node \"ip-10-0-131-170.ec2.internal\" DevicePath \"\""
Sep 17 22:26:06.883764 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.883730    2587 generic.go:356] "Generic (PLEG): container finished" podID="3b29350a-c3ee-464e-aa6f-c250c68bd756" containerID="1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd" exitCode=0
Sep 17 22:26:06.883924 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.883772    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"
Sep 17 22:26:06.883924 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.883819    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" event={"ID":"3b29350a-c3ee-464e-aa6f-c250c68bd756","Type":"ContainerDied","Data":"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"}
Sep 17 22:26:06.883924 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.883866    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw" event={"ID":"3b29350a-c3ee-464e-aa6f-c250c68bd756","Type":"ContainerDied","Data":"ebcf97bc6d5e4b4f7acaad775770a9b59c21ee89f010680ce8d9f866e6e5e2bd"}
Sep 17 22:26:06.883924 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.883889    2587 scope.go:117] "RemoveContainer" containerID="1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"
Sep 17 22:26:06.892707 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.892694    2587 scope.go:117] "RemoveContainer" containerID="1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"
Sep 17 22:26:06.892971 ip-10-0-131-170 kubenswrapper[2587]: E0917 22:26:06.892953    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd\": container with ID starting with 1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd not found: ID does not exist" containerID="1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"
Sep 17 22:26:06.893033 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.892983    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd"} err="failed to get container status \"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd\": rpc error: code = NotFound desc = could not find container \"1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd\": container with ID starting with 1e38761235b0fbd7e85f57fa60a1e20db5548950613fe1fd221ab8f21f55cecd not found: ID does not exist"
Sep 17 22:26:06.902553 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.902528    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 22:26:06.907128 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:06.907108    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-d48fb5469-zq6hw"]
Sep 17 22:26:08.857446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:08.857413    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b29350a-c3ee-464e-aa6f-c250c68bd756" path="/var/lib/kubelet/pods/3b29350a-c3ee-464e-aa6f-c250c68bd756/volumes"
Sep 17 22:26:22.334556 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:22.334525    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd68bd744-hc96m_7efd9b25-1440-45ed-b0f6-57d969bf6351/authorino/0.log"
Sep 17 22:26:28.020925 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:28.020887    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b978497bd-sncsz_aed3d9d8-4510-41e7-b587-a48175875141/manager/0.log"
Sep 17 22:26:28.660204 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:28.660175    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555cdfd7b4-x9qtq_325bb420-b689-43ef-ae0a-9aa4565121b7/copy-manifests/0.log"
Sep 17 22:26:28.683277 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:28.683245    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555cdfd7b4-x9qtq_325bb420-b689-43ef-ae0a-9aa4565121b7/manager/0.log"
Sep 17 22:26:29.177802 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:29.177770    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c966cd48f-jxh7l_1d217209-36d7-4eb7-ac32-18025f84e247/maas-api/0.log"
Sep 17 22:26:29.405443 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:29.405413    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29828055-jp48k_ada5735b-e1f1-4702-bd01-b4e448aabd14/cleanup/0.log"
Sep 17 22:26:29.518275 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:29.518243    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wqm9q_fb686664-145b-4771-936f-3f70fb28884d/postgres/0.log"
Sep 17 22:26:30.334732 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:30.334694    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd68bd744-hc96m_7efd9b25-1440-45ed-b0f6-57d969bf6351/authorino/0.log"
Sep 17 22:26:30.691180 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:30.691102    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-psg95_05748130-db33-4f47-9b2d-3f3e9c28f895/kuadrant-console-plugin/0.log"
Sep 17 22:26:31.812802 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:31.812767    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f7577f47d-r96lt_5393fa71-d7a8-4c13-855b-f52c839dcee2/kube-auth-proxy/0.log"
Sep 17 22:26:32.187201 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:32.187084    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"
Sep 17 22:26:32.769348 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:32.769316    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mb2fr_26786f2f-3a0b-4bfe-bfb2-d00dc915f968/main/0.log"
Sep 17 22:26:33.121747 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:33.121645    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzwvnd_0b74b9b7-41e5-488b-84c4-fda917e61859/main/0.log"
Sep 17 22:26:33.353065 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:33.353034    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-cw6j4_69c2dd6a-7aee-43d6-b2f1-4896088dd92a/main/0.log"
Sep 17 22:26:39.027307 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:39.027258    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:26:39.034183 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:39.034160    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9vm42_261a5def-2288-4e8a-82bf-ba2a02d8482f/ovn-acl-logging/0.log"
Sep 17 22:26:42.359398 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:42.359366    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-srdl6_7896882d-59fe-4d40-a42f-6915c7355f43/global-pull-secret-syncer/0.log"
Sep 17 22:26:42.572540 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:42.572489    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jq6hw_220a0dbe-bcb0-4145-9aef-9422d1200df8/konnectivity-agent/0.log"
Sep 17 22:26:42.629507 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:42.629411    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-170.ec2.internal_9817c5934fa5926d73822e024f014f2a/haproxy/0.log"
Sep 17 22:26:47.438480 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:47.438390    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fd68bd744-hc96m_7efd9b25-1440-45ed-b0f6-57d969bf6351/authorino/0.log"
Sep 17 22:26:47.568368 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:47.568343    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-psg95_05748130-db33-4f47-9b2d-3f3e9c28f895/kuadrant-console-plugin/0.log"
Sep 17 22:26:49.216632 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.216596    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/alertmanager/0.log"
Sep 17 22:26:49.281937 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.281889    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/config-reloader/0.log"
Sep 17 22:26:49.370990 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.370947    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/kube-rbac-proxy-web/0.log"
Sep 17 22:26:49.434011 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.433982    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/kube-rbac-proxy/0.log"
Sep 17 22:26:49.465611 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.465582    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/kube-rbac-proxy-metric/0.log"
Sep 17 22:26:49.491879 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.491850    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/prom-label-proxy/0.log"
Sep 17 22:26:49.541968 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.541934    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da97ee6d-d654-473e-a638-217f97be9213/init-config-reloader/0.log"
Sep 17 22:26:49.633046 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.633011    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-w48p9_dd8f947d-fa50-43f8-a69f-2088205b7e22/kube-state-metrics/0.log"
Sep 17 22:26:49.655272 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.655243    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-w48p9_dd8f947d-fa50-43f8-a69f-2088205b7e22/kube-rbac-proxy-main/0.log"
Sep 17 22:26:49.683452 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.683423    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-w48p9_dd8f947d-fa50-43f8-a69f-2088205b7e22/kube-rbac-proxy-self/0.log"
Sep 17 22:26:49.719596 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.719571    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f8744fd5b-2szb4_2f20beab-396f-4f3a-8791-c8f875d65750/metrics-server/0.log"
Sep 17 22:26:49.760745 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.760664    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-rj7kn_63bd4e54-04c1-4419-b9b1-46c06a15c107/monitoring-plugin/0.log"
Sep 17 22:26:49.981749 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:49.981721    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nh28r_321d4648-7879-4cc9-8721-1c8562caba71/node-exporter/0.log"
Sep 17 22:26:50.019624 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.019546    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nh28r_321d4648-7879-4cc9-8721-1c8562caba71/kube-rbac-proxy/0.log"
Sep 17 22:26:50.049661 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.049633    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nh28r_321d4648-7879-4cc9-8721-1c8562caba71/init-textfile/0.log"
Sep 17 22:26:50.193922 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.193886    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-2kk5n_0b48fa00-e1d5-410f-ad2b-065b561197b0/kube-rbac-proxy-main/0.log"
Sep 17 22:26:50.223709 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.223674    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-2kk5n_0b48fa00-e1d5-410f-ad2b-065b561197b0/kube-rbac-proxy-self/0.log"
Sep 17 22:26:50.255884 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.255851    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-2kk5n_0b48fa00-e1d5-410f-ad2b-065b561197b0/openshift-state-metrics/0.log"
Sep 17 22:26:50.302672 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.302558    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/prometheus/0.log"
Sep 17 22:26:50.330743 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.330718    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/config-reloader/0.log"
Sep 17 22:26:50.359749 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.359723    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/thanos-sidecar/0.log"
Sep 17 22:26:50.388730 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.388705    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/kube-rbac-proxy-web/0.log"
Sep 17 22:26:50.415876 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.415854    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/kube-rbac-proxy/0.log"
Sep 17 22:26:50.444522 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.444493    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/kube-rbac-proxy-thanos/0.log"
Sep 17 22:26:50.471677 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.471654    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_864a8562-9dea-41b9-ac87-b17483c0dfd9/init-config-reloader/0.log"
Sep 17 22:26:50.508156 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.508129    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-vprm8_e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1/prometheus-operator/0.log"
Sep 17 22:26:50.531038 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.531011    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-vprm8_e97ac2d7-9c3e-4eae-b95f-de9b2fe97ff1/kube-rbac-proxy/0.log"
Sep 17 22:26:50.572427 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.572334    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-q6wtx_5841ba25-b480-4ba5-bdc3-10d4e8a5dabe/prometheus-operator-admission-webhook/0.log"
Sep 17 22:26:50.753703 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.753673    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/thanos-query/0.log"
Sep 17 22:26:50.803160 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.803128    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/kube-rbac-proxy-web/0.log"
Sep 17 22:26:50.869762 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.869682    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/kube-rbac-proxy/0.log"
Sep 17 22:26:50.930047 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.930020    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/prom-label-proxy/0.log"
Sep 17 22:26:50.964446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.964414    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/kube-rbac-proxy-rules/0.log"
Sep 17 22:26:50.997305 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:50.997258    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747fb7c5cc-qqn9g_376264b5-347d-40d4-a1fb-ca58256eba35/kube-rbac-proxy-metrics/0.log"
Sep 17 22:26:52.647446 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.647410    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"]
Sep 17 22:26:52.647877 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.647819    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b29350a-c3ee-464e-aa6f-c250c68bd756" containerName="maas-api"
Sep 17 22:26:52.647877 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.647832    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b29350a-c3ee-464e-aa6f-c250c68bd756" containerName="maas-api"
Sep 17 22:26:52.647972 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.647914    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b29350a-c3ee-464e-aa6f-c250c68bd756" containerName="maas-api"
Sep 17 22:26:52.652580 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.652561    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.655419 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.655400    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmg5p\"/\"kube-root-ca.crt\""
Sep 17 22:26:52.655511 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.655470    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmg5p\"/\"openshift-service-ca.crt\""
Sep 17 22:26:52.656693 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.656675    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jmg5p\"/\"default-dockercfg-gg4dc\""
Sep 17 22:26:52.659022 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.658992    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-lib-modules\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.659146 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.659040    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-podres\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.659146 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.659109    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-proc\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.659585 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.659138    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-sys\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.659585 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.659276    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tb4m\" (UniqueName: \"kubernetes.io/projected/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-kube-api-access-8tb4m\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.662627 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.662599    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"]
Sep 17 22:26:52.760242 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760199    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-lib-modules\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760253    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-podres\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760323    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-proc\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760351    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-sys\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760424    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-lib-modules\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760430    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-podres\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760462 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760452    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-proc\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760668 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760453    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tb4m\" (UniqueName: \"kubernetes.io/projected/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-kube-api-access-8tb4m\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.760668 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.760487    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-sys\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.770538 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.770506    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tb4m\" (UniqueName: \"kubernetes.io/projected/dc55c45c-1dab-46fc-b873-f5c3092f5bbd-kube-api-access-8tb4m\") pod \"perf-node-gather-daemonset-k2z8x\" (UID: \"dc55c45c-1dab-46fc-b873-f5c3092f5bbd\") " pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:52.963484 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:52.963396    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:53.238723 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:53.238694    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"]
Sep 17 22:26:53.240488 ip-10-0-131-170 kubenswrapper[2587]: W0917 22:26:53.240462    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddc55c45c_1dab_46fc_b873_f5c3092f5bbd.slice/crio-b658d7555137b10514af173d9b8f7ef8fc55c9a15c003c44070ae1c07f6dbe9f WatchSource:0}: Error finding container b658d7555137b10514af173d9b8f7ef8fc55c9a15c003c44070ae1c07f6dbe9f: Status 404 returned error can't find the container with id b658d7555137b10514af173d9b8f7ef8fc55c9a15c003c44070ae1c07f6dbe9f
Sep 17 22:26:53.634779 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:53.634695    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-sgjbq_7a9b572e-494b-42de-b511-9a3f56ae5b95/download-server/0.log"
Sep 17 22:26:54.130246 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:54.130208    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x" event={"ID":"dc55c45c-1dab-46fc-b873-f5c3092f5bbd","Type":"ContainerStarted","Data":"2d25e11d353d462a475f1647f02ef168ef3788bd7e8aaacff63b9bc3b7f1acbc"}
Sep 17 22:26:54.130734 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:54.130320    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x" event={"ID":"dc55c45c-1dab-46fc-b873-f5c3092f5bbd","Type":"ContainerStarted","Data":"b658d7555137b10514af173d9b8f7ef8fc55c9a15c003c44070ae1c07f6dbe9f"}
Sep 17 22:26:54.130734 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:54.130362    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:26:54.149117 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:54.149066    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x" podStartSLOduration=2.149048363 podStartE2EDuration="2.149048363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 22:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:26:54.148168017 +0000 UTC m=+2717.496901902" watchObservedRunningTime="2026-09-17 22:26:54.149048363 +0000 UTC m=+2717.497782252"
Sep 17 22:26:55.574492 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:55.574459    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fldpp_ce72014b-3d76-4b31-bdd3-4d7147f82989/dns/0.log"
Sep 17 22:26:55.605967 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:55.605943    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fldpp_ce72014b-3d76-4b31-bdd3-4d7147f82989/kube-rbac-proxy/0.log"
Sep 17 22:26:55.748256 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:55.748223    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz4gv_9d2ab88e-a956-41ad-b84f-02f349dea2c2/dns-node-resolver/0.log"
Sep 17 22:26:56.423587 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:56.423544    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kpg5l_43badf67-82f0-414c-82d2-84bf1b7ac9a9/node-ca/0.log"
Sep 17 22:26:58.585398 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:58.585359    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f7577f47d-r96lt_5393fa71-d7a8-4c13-855b-f52c839dcee2/kube-auth-proxy/0.log"
Sep 17 22:26:58.764820 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:58.764789    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"
Sep 17 22:26:59.793421 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:26:59.793387    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6t4gt_47a8d994-d592-4677-ba9c-5b63e154d5cc/serve-healthcheck-canary/0.log"
Sep 17 22:27:00.146609 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:00.146532    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jmg5p/perf-node-gather-daemonset-k2z8x"
Sep 17 22:27:00.577983 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:00.577955    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sf2l7_3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8/kube-rbac-proxy/0.log"
Sep 17 22:27:00.613687 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:00.613663    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sf2l7_3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8/exporter/0.log"
Sep 17 22:27:00.641200 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:00.641171    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sf2l7_3d0f8fc4-c81e-42a7-b6c0-ee76cbe08be8/extractor/0.log"
Sep 17 22:27:03.927364 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:03.927280    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b978497bd-sncsz_aed3d9d8-4510-41e7-b587-a48175875141/manager/0.log"
Sep 17 22:27:04.491916 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:04.491883    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555cdfd7b4-x9qtq_325bb420-b689-43ef-ae0a-9aa4565121b7/manager/0.log"
Sep 17 22:27:04.521070 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:04.521044    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555cdfd7b4-x9qtq_325bb420-b689-43ef-ae0a-9aa4565121b7/copy-manifests/0.log"
Sep 17 22:27:04.585272 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:04.585237    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f7577f47d-r96lt_5393fa71-d7a8-4c13-855b-f52c839dcee2/kube-auth-proxy/0.log"
Sep 17 22:27:04.676910 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:04.676876    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"
Sep 17 22:27:05.978183 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:05.978150    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7746dd6866-wjvz6_834453c9-0bae-47c8-b9f8-b306f3d47d64/manager/0.log"
Sep 17 22:27:07.197505 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:07.197480    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f7577f47d-r96lt_5393fa71-d7a8-4c13-855b-f52c839dcee2/kube-auth-proxy/0.log"
Sep 17 22:27:07.293836 ip-10-0-131-170 kubenswrapper[2587]: I0917 22:27:07.293799    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hzhpk_a0f27f4a-5e01-4eac-8cad-f99a3908ded8/payload-processing/0.log"